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EXECUTIVE  SUMMARY 


In  2013  and  2015,  CH2M  HILL,  Inc.  (CH2M)  performed  two  Environmental  Security  Technology 
Certification  Program  (ESTCP)  Munitions  Response  Live  Site  Demonstrations  as  part  of  project 
MR-201314.  The  2013  demonstration  was  performed  at  the  Shooting  Fields  (MU705),  located  at 
New  Boston  Air  Force  Station  (NBAFS),  New  Hampshire  (NH).  The  2015  demonstration  was 
performed  under  the  existing  project  at  Munitions  Response  Site  (MRS)  R-04A  West,  at 
Tobyhanna  Army  Depot  (TOAR)  Formerly  Used  Defense  Site  (FUDS),  in  Pennsylvania. 

CH2M  performed  the  demonstrations  at  NBAFS  and  TOAR  using  the  Naval  Research  Laboratory 
(NRL)  Time-domain  Electromagnetic  Towed  Array  Detection  System  2x2  (TEMTADS),  an 
advanced  transient  electromagnetic  (TEM)  system  designed  for  the  detection  and  classification  of 
munitions  and  explosives  of  concern  (MEC)  (specifically  discarded  military  munitions  [DMM] 
and  unexploded  ordnance [UXO]).  Additional  data  was  collected  at  each  of  these  sites  by  Black 
Tusk  Geophysics  using  the  Man  Portable  Vector  (MPV)  but  those  results  are  not  addressed  in  this 
report. 

The  demonstration  at  NBAFS  was  designed  to  investigate  the  classification  methodology  at  a  site 
suspected  to  contain  a  high  density  of  subsurface  metallic  objects,  as  well  as  a  variety  of  munitions 
such  as  20-millimeter  (mm)  projectiles.  The  TEMTADS  was  employed  in  both  dynamic  and  cued 
modes  using  the  sensor’s  standard  wheel  configuration.  Positioning  was  achieved  through  the  use 
of  a  real  time  kinematic  (RTK)  global  positioning  system  (GPS). 

Approximately  6.1  acres  were  mapped  with  the  TEMTADS  system  in  dynamic  mode.  The 
density  of  geophysical  anomalies  identified  from  the  survey  was  an  order  of  magnitude  higher 
than  anticipated.  In  order  to  stay  within  the  budgeted  scope  of  the  demonstration,  1,500 
anomalies  from  a  subarea  of  the  site  were  selected  for  cued  interrogation  with  the  same  system. 
Data  analysis  of  the  cued  measurements  confirmed  a  high  density  of  metal  with  a 
large  percentage  of  the  anomaly  locations  being  multi-target  scenarios.  The  project  team 
(ESTCP,  NBAFS,  and  CH2M)  was  haulted  after  a  several  week  review  of  intrusive  results, 
which  determinined  that  classification  at  the  site  would  1)  not  result  in  cost  savings  as  opposed 
to  intrusively  investigating  all  anomalies  and  2)  be  ineffective.  This  decision  was  based  on  the 
fact  that  70  percent  of  the  intrusive  investigations  resulted  in  discovery  of  a  target  of  interest 
(TOI)  and  each  dig  typically  yielded  multiple  items.  Approximately  33  percent  of  the  anomalies 
investigated  resulted  in  three  or  more  sources  and,  under  these  circumstances,  the  inversion 
results  do  not  appear  to  accurately  represent  the  ground  truth. 

The  demonstration  at  TOAR  was  designed  to  investigate  the  use  of  advanced  electromagnetic 
induction  (EMI)  sensors  at  a  densely  wooded  site  with  challenging  micro-terrain  features  (e.g., 
impact  craters,  rocks,  boulders,  and  gullies)  for  detecting  munitions  down  to  the  size  of  37-mm 
projectiles.  The  TEMTADS  was  employed  in  both  dynamic  and  cued  modes  in  two-person  litter 
mode  and  positioning  was  achieved  through  the  use  of  a  Robotic  Total  Station  (RTS). 

Approximately  0.71  acre  was  mapped  with  the  TEMTADS  system.  Production  was  significantly 
hindered  by  the  remote  location  and  site  conditions  and  coverage  for  only  one  of  the  four  grids  initially 
selected  for  investigation  was  at  100  percent  (not  including  gaps  because  of  physical  obstructions). 
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Dynamic  surveying  is  very  much  a  function  of  terrain,  navigation  software,  and  deployment 
platform.  Because  of  the  challenging  conditions  at  TOAR,  production  rates  achieved  as  part  of  the 
demonstration  are  estimated  to  be  much  lower  than  production  rates  at  sites  with  more  favorable 
conditions. 

A  total  of  429  anomalies  were  identified  by  the  TEMTADS  dynamic  data  analysis.  These  anomaly 
locations,  and  an  additional  68  targets  selected  from  the  MPV  dynamic  data,  were  interrogated 
(cued  data  collection)  with  the  TEMTADS  system.  The  cued  data  were  used  to  classify  each  target 
as  being  a  TOI  (dig)  or  high-likelihood  non-TOI  (do  not  dig).  The  TEMTADS  data  collection  and 
analysis  resulted  in  successful  detection  and  classification  of  all  known  TOIs  (seeds  and  native). 

The  estimated  cost  of  deploying  the  TEMTADS  based  on  the  demonstrations  at  NBAFS  and 
TOAR  is  high,  approximately  two  or  three  times  the  cost  of  deploying  commercially  available 
systems  such  as  the  Geonics  EM61-MK2.  The  primary  demonstrated  advantage  of  deploying  the 
TEMTADS  lies  with  the  ability  to  use  the  data  collected  to  classify  targets  and  thus  to  produce  a 
prioritized  dig  list.  At  TOAR,  the  use  of  a  prioritized  dig  list  would  have  reduced  the  number  of 
digs  by  71  percent  or  more  (depending  on  the  stop  dig  threshold).  This  would  result  in  an  overall 
cost  savings  of  approximately  30  percent  in  the  cost  of  the  intrusive  investigation.  For  sites  such 
as  TOAR,  where  targets  are  successfully  discriminated,  the  higher  cost  associated  with  deployment 
of  the  TEMTADS  is  justified  on  the  basis  that  it  can  substantially  reduce  the  cost  of  digging.  For 
sites  such  as  NBAFS,  where  the  anomaly  density  is  very  high  and  detecting  munitions  down  to  the 
size  of  20-mm  projectiles  is  required,  use  of  the  TEMTADS  is  not  warranted. 
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1.0  INTRODUCTION 


This  demonstration  cost  and  performance  report  details  the  Environmental  Security  Technology 
Certification  Program  (ESTCP)  Munitions  Response  Live  Site  Demonstrations  at  New  Boston  Air 
Force  Station  (NBAFS),  performed  in  the  summer  of  2013,  and  at  Munitions  Response  Site  (MRS) 
R-04A  West,  at  Tobyhanna  Army  Depot  (TOAR)  Formerly  Used  Defense  Site  (FUDS),  in 
Pennsylvania,  performed  in  the  summer  of  2015. 

At  both  sites  the  Naval  Research  Laboratory’s  (NRL)  man-portable  version  of  the  Time-domain 
Electromagnetic  Towed  Array  Detection  System  2x2  (TEMTADS)  was  employed  in  both  dynamic 
and  cued  modes.  Positioning  was  achieved  through  use  of  a  real-time  kinematic  (RTK)  global 
positioning  system  (GPS)  at  NBAFS  and  a  Robotic  Total  Station  (RTS)  at  TOAR.  (Additional 
data  were  collected  at  each  of  these  sites  by  Black  Tusk  Geophysics  using  the  Man  Portable  Vector 
[MPV]  but  those  results  are  not  addressed  in  this  report.)  The  demonstrations  were  performed  in 
accordance  with  the  ESTCP  Munitions  Response  Live  Site  Demonstrations,  New  Boston  Air  Force 
Station,  NH,  Demonstration  Plan  (ESTCP,  2013)  and  the  Live  Site  Demonstrations  TEMTADS 
2x2  Demonstration  Plan  MRS-R04A  (West),  Tobyhanna  Army  Depot  FUDS,  Pennsylvania, 
ESTCP  Project  MR-201314  (ESTCP,  2015),  respectively. 

1.1  BACKGROUND 

A  large  quantity  of  FUDS  and  Base  Realignment  and  Closure  land  has  been  contaminated  with 
munitions  and  explosives  of  concern  (MEC),  and  specifically  discarded  military  munitions 
(DMM)  and/or  unexploded  ordnance  (UXO).  Changes  in  the  land  use  designation  for  these  areas 
often  requires  a  munitions  response  remedial  action  to  remove  hazardous  items  before  reuse.  Using 
conventional  technology  and  procedures,  remediation  costs  are  driven  by  the  need  to  visually 
characterize  all  metal  targets;  however,  most  targets  are  non-hazardous  scrap  with  only  a 
small  percentage  actually  being  classified  as  dangerous.  Significant  overall  cost  savings  can  be 
achieved  using  technology  that  reliably  classifies  metal  targets  and  substantially  reduces  the 
number  of  required  intrusive  investigations.  The  demonstrations  at  NBAFS  and  TOAR  were 
intended  to  help  constrain  the  environments  in  which  advanced  geophysical  classification  (AGC) 
technology  could  be  effectively  applied  at  sites  with  challenging  conditions  and  still  result  in  a 
significant  reduction  in  intrusive  activities. 

CH2M  HILL,  Inc.  (CH2M)  served  as  contractor  to  ESTCP  under  contract  W912HQ-13-C-0039 
for  the  demonstrations  at  NBAFS  and  TOAR.  The  demonstration  at  NBAFS  was  designed  to 
investigate  the  use  of  the  TEMTADS  sensor  at  a  site  that  contained  a  high  density  of  metal  in  the 
sub-surface  and  a  large  variety  of  TOI  down  to  20-millimeter  (mm)  projectiles.  The  demonstration 
at  TOAR  was  designed  to  investigate  the  use  of  the  TEMTADS  sensor  at  a  densely  wooded  site 
with  challenging  micro-terrain  features  (e.g.,  impact  craters,  rocks,  boulders,  gullies)  for  detection 
of  37-mm  projectiles.  As  a  result  of  the  site  conditions  at  TOAR,  the  TEMTADS  was  operated  in 
two-person  litter  carry  mode  during  dynamic  (i.e.,  detection)  data  collection. 
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1.2  OBJECTIVES  OF  THE  DEMONSTRATION 


The  overall  objective  of  both  demonstrations  was  to  demonstrate  the  overall  detection  and 
classification  performance  of  the  TEMTADS;  however,  the  site  specific  objectives  of  the  two 
demonstrations  where  slightly  different. 

1.2.1  NBAFS 

At  NBAFS,  the  overall  objective  was  to  demonstrate  the  detection  and  classification  performance 
of  the  sensor  at  a  site  with  a  heavy  density  of  metal  and  targets  of  interest  (TOI)  as  small  as  20-mm 
projectiles.  CH2M  performed  the  following  tasks  in  order  to  achieve  this  overall  objective. 

•  Placement  of  subsurface  quality  control  (‘blind’)  seeds  within  the  demonstration  site 

•  Establishment  of  an  instrument  verification  strip  (IVS) 

•  Collection  of  dynamic  data  using  the  TEMTADS 

•  Processing  of  dynamic  data  and  selection  of  targets 

•  Cued  interrogation  of  targets  using  the  TEMTADS 

•  Processing  of  cued  geophysical  data 

•  Reacquisition  and  intrusive  investigation  of  targets 

1.2.2  TOAR 

At  TOAR,  the  overall  objective  was  to  demonstrate  the  detection  and  classification  performance 
of  the  sensor  in  challenging  terrain  conditions  and  for  TOI  as  small  as  37-mm  projectiles.  A 
secondary  objective  was  to  provide  data  to  ESTCP  for  comparison  of  the  TEMTADS  capabilities 
under  these  conditions  to  those  of  the  MPV.  (Note  that  this  report  covers  the  TEMTADS  operations 
and  analysis  only;  comparison  of  the  results  achieved  with  those  of  the  MPV  was  performed  by 
the  ESTCP  Program  Office).  To  achieve  these  overall  objectives  CH2M  performed  the  following 
tasks. 


•  Collection  of  dynamic  transect  data  across  using  a  Geometries  G-858G  magnetometer  to 
identify  an  appropriate  demonstration  area 

•  Selection  of  the  demonstration  grids  based  on  G-858G  data 

•  Reduction  of  vegetation 

•  Placement  of  subsurface  quality  control  (‘blind’)  seeds  within  the  demonstration  site 

•  Establishment  of  an  IVS 

•  Collection  of  dynamic  data  cueing  the  TEMTADS 

•  Interrogation  of  cued  targets  using  the  TEMTADS 

•  Processing  of  dynamic  and  cued  geophysical  data 

•  Reacquisition  and  intrusive  investigation  of  targets  selected  for  cued  interrogation 
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1.3  REGULATORY  DRIVERS 


The  Military  Munitions  Response  Program  (MMRP)  is  charged  with  characterizing  and,  where 
necessary,  remediating  MRSs.  When  a  MRS  is  remediated,  it  is  typically  mapped  with  a 
geophysical  system,  based  on  either  a  magnetometer  or  EMI  sensor,  and  the  locations  of  all 
detectable  signals  are  investigated.  Many  of  these  detections  do  not  correspond  to  munitions,  but 
rather  to  other  harmless  metallic  objects  or  geology.  Field  experience  indicates  that  often  in  excess 
of  90  percent  of  objects  excavated  during  the  course  of  a  munitions  response  are  found  to  be 
nonhazardous  items.  Current  geophysical  technology,  as  it  is  traditionally  implemented,  does  not 
provide  a  physics-based,  quantitative,  validated  means  to  discriminate  between  hazardous 
munitions  and  nonhazardous  items. 

With  no  information  to  suggest  the  origin  of  the  signals,  all  anomaly  sources  are  currently  treated 
as  though  they  are  intact  munitions  when  they  are  investigated.  They  are  carefully  excavated  by 
certified  UXO  technicians  using  a  process  that  often  requires  expensive  safety  measures,  such  as 
barriers  or  exclusion  zones.  As  a  result,  most  of  the  costs  to  remediate  a  munitions-impacted  site 
are  currently  spent  on  excavating  items  that  pose  no  threat.  If  these  items  could  be  determined  with 
high  confidence  to  be  nonhazardous,  some  of  these  expensive  measures  could  be  eliminated  or  the 
items  could  be  left  unexcavated  entirely. 

The  MMRP  is  severely  constrained  by  available  resources.  Remediation  of  the  entire  inventory 
using  current  practices  is  cost  prohibitive  within  current  and  anticipated  funding  levels.  With 
current  planning,  estimated  completion  dates  for  munitions  response  on  many  sites  are  decades 
out.  The  United  States  Department  of  Defense’ s  Defense  Science  Board  observed  in  its  2003  report 
that  significant  cost  savings  could  be  realized  if  successful  classification  of  munitions  and  other 
nonhazardous  sources  of  anomalies  could  be  implemented.  If  these  savings  were  realized,  the 
limited  resources  of  the  MMRP  could  be  used  to  accelerate  the  remediation  of  MRSs  that  are 
currently  forecasted  to  be  untouched  for  decades. 

While  no  regulatory  barriers  exist  barring  the  application  of  advanced  EMI  technology  (such  as 
the  TEMTADS)  at  sites  contaminated  with  DMM/UXO,  convincing  regulators  and  other 
stakeholders  of  the  reliability  of  advanced  EMI  systems  classification  of  metal  targets  is  often 
challenging.  The  demonstrations  at  NBAFS  and  TOAR  were  designed  to  help  the  ESTCP  Program 
Office  address  these  challenges  by  evaluating  the  capabilities  of  the  TEMTADS  at  sites  with 
challenging  conditions. 
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2.0  TECHNOLOGY 


CH2M’s  demonstrations  at  NBAFS  and  TOAR  were  designed  to  investigate  the  use  and  document 
the  capabilities  of  the  TEMTADS  advanced  EMI  sensor  at  sites  with  challenging  conditions  (e.g., 
dense  woods,  high  anomaly  density,  small  TOI,  and  varying  micro-terrain  features)  through 
dynamic  and  cued  data  collection.  Analysis  of  the  data  was  performed  using  conventional  and 
AGC-specific  data  processing  methods. 

2.1  TECHNOLOGY  DESCRIPTION 

2.1.1  TEMTADS  2x2 

The  TEMTADS  is  comprised  of  four  individual  sensor  elements  arranged  in  a  2-by-2  array.  Each 
element  consists  of  a  transmit  coil  and  a  three-axis  receiver  cube.  Each  cube  has  dimensions  of 
8  centimeters  (cm).  The  center-to-center  distance  is  40  cm,  yielding  an  array  measuring  80  cm  by 
80  cm.  A  TEMTADS  sensor  element  under  construction,  the  sensor  array  (with  protective  cover 
removed),  and  a  schematic  diagram  of  the  array  are  shown  as  Figure  2-1.  Decay  data  are  collected 
with  a  500-kilohertz  sample  rate  until  25  milliseconds  (ms)  after  turn  off  of  the  excitation  pulse. 
This  results  in  a  raw  decay  of  12,500  points,  which  is  too  many  to  be  used  practically.  These  raw 
decay  measurements  are  grouped  into  122  logarithmically  spaced  ‘gates’  with  center  times  ranging 
from  25  microseconds  to  24.35  ms  with  5  percent  widths  and  are  saved  to  disk. 


Figure  2-1.  Single-element  TEMTADS  Three-axis  Receiver  Transmit  Coil  (Left  Photo), 
Sensor  Array  (Right  Photo),  and  Schematic  Array  Diagram 

The  TEMTADS  is  a  person-portable  system  that  is  deployed  on  a  set  of  standard  wheels.  This 
results  in  a  sensor-to-ground  offset  of  approximately  1 8  cm,  or  results  in  a  two-person  litter  mode 
where  the  carry  height  is  approximately  the  same  as  when  deployed  in  wheel  mode.  The  array 
structure  is  fabricated  from  PVC  plastic  and  Garolite  fiberglass.  The  transmitter  electronics  and 
the  data  acquisition  computer  are  mounted  on  the  operator  backpack,  a  positioning  unit  is  mounted 
on  top  of  a  Garolite  fiberglass  pole,  and  an  inertial  measurement  unit  (IMU)  is  mounted  above  the 
array.  The  TEMTADS  can  be  operated  in  two  modes:  dynamic  and  cued.  Data  collection  is 
controlled  in  dynamic  mode  using  G&G  Science’s  EM3D  application  suite.  In  cued  mode,  the 
locations  of  previously  identified  anomalies  are  flagged  and  surveyed  with  static  measurements 
directly  over  the  flag  location. 

Custom  software  written  by  the  NRL  is  used  to  control  the  cued  data  acquisition.  Both  sets  of 
software  are  accessed  through  a  remote  desktop  on  a  computer  tablet. 
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2.1.2  Positioning  Systems 

Positioning  data  were  recorded  using  a  RTK  GPS  at  NBAFS  and  a  RTS  at  TOAR.  Different 
positioning  systems  were  necessary  because  of  the  difference  in  tree  cover  at  the  two  sites. 

The  survey  area  at  NBAFS  was  open  (trees  were  present  along  the  edges  of  the  survey)  allowing 
positional  data  to  be  recorded  using  a  Trimble  R8  receiver  operating  in  RTK  mode.  The  GPS 
receiver  was  centered  over  the  TEMTADS  sensor.  Figure  2-2  shows  a  portion  of  the  survey  area 
(left  photograph)  and  the  sensor  with  the  GPS  antenna  set-up  for  surveying  (right  photograph). 
Real-time  corrections  were  provided  using  a  local  base  station  consisting  of  a  separate  R8  receiver 
and  Trimble  TDL450  external  radio.  The  base  station  and  external  radio  set-up  are  shown  on 
Figure  2-3.  The  pitch,  roll,  and  yaw  of  the  system  were  recorded  using  an  IMU  mounted  beneath 
the  GPS  antenna. 


Figure  2-2.  NBAFS  Survey  Area  Conditions  and  TEMTADS  with  GPS  Antenna 
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Figure  2-3.  NBAFS  RTK  GPS  Base  Station 

At  TOAR,  site  conditions  did  not  permit  the  reliable  use  of  a  RTK  GPS  positioning  system. 
Positional  data  were  recorded  using  a  Trimble  R7  RTS  with  a  MT1000  active  prism  centered  over 
the  TEMTADS  sensor.  A  non-metallic  pole  was  added  to  the  system  to  raise  the  height  of  the 
prism  above  the  heads  of  the  operators,  thus  allowing  the  operators  to  walk  without  blocking  the 
prism  from  the  RTS  base  station.  Positional  data  were  logged  at  a  nominal  rate  of  10  hertz.  The 
pitch,  roll,  and  yaw  of  the  cart  were  recorded  using  an  IMU  mounted  beneath  the  prism.  Figure  2-4 
shows  the  wooded  site  conditions  of  the  TOAR  survey  area,  the  TEMTADS  sensor  with  the 
MT1000  prism  deployed  for  surveying  (left  photograph),  and  the  RTS  base  station  (right 
photograph). 
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Figure  2-4.  TOAR  Survey  Area  Conditions,  TEMTADS  with  MT1000  Prism,  and  RTK 

Base  Station 


2.1.3  Data  Acquisition  User  Interface 

At  both  NBAFS  and  TOAR,  data  collection  was  controlled  using  a  tablet,  which  wirelessly  (IEEE 
802.  llg)  communicated  with  the  data  acquisition  computer  on  the  operator  backpack.  The  tablet 
operator  also  managed  field  notes  and  team  orienteering  functions.  Images  of  the  tablet  user 
interface  for  dynamic  and  cued  data  acquisition  are  shown  in  Figure  2-5. 


Figure  2-5.  Dynamic  Data  Acquisition  Interface  (Right)  and  Cued  Data  Acquisition 

Interface  (Feft) 

2.2  ADVANTAGES  AND  EIMITATIONS  OF  THE  TECHNOFOGY 

Site  conditions  at  NBAFS  supported  the  efficient  use  and  deployment  of  the  TEMTADS  sensor  in  its 
standard  wheel  mode.  Although  the  TEMTADS  was  designed  to  be  used  in  difficult  terrain  and  in 
wooded  areas  that  larger  systems  (e.g.,  MetalMapper)  could  not  access,  it  was  anticipated  that  the 
micro-terrain  features  at  TOAR  could  result  in  increased  sensor  noise  levels  because  of  the  bouncing 
and  shaking  of  the  array  as  its  skids  made  contact  with  the  ground  surface,  rocks,  and  fallen  logs. 
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Irregular  heights  because  of  its  being  lifted  over  objects  can  also  decrease  sensor  sensitivity.  In 
addition,  imprecision  in  the  positioning  data  can  result  from  increased  pitch  and  roll  movements 
exacerbated  by  increased  height  of  the  RTS  prism.  As  a  result,  one  of  the  objectives  of  the  TOAR 
demonstration  was  to  determine  how  well  the  TEMTADS  performs  in  litter  carry  mode  and  to 
assess  the  efficacy  of  this  modality  with  difficult  survey  conditions. 

Another  serious  limitation  of  the  technology  is  anomaly  density;  for  all  advanced  EMI  systems 
there  is  a  limiting  anomaly  density  above  which  the  response  of  individual  targets  cannot  be 
separated.  Recent  developments  in  multi-source  solvers  have  facilitated  improved  results  within 
elevated  anomaly  density  areas,  although  for  sites  with  areas  of  significant  anomaly  density  the 
result  may  still  be  saturated  response  areas  that  cannot  be  subjected  to  classification.  At  TOAR,  an 
exploratory  G-858G  (magnetometer)  survey  was  conducted  to  identify  areas  of  suitable  density 
for  the  demonstration.  At  NBAFS,  one  of  the  overall  objectives  of  the  demonstration  was  to  assess 
the  efficacy  of  the  TEMTADS  in  performing  dynamic  detection  and  cued  classification  at  a  site 
known  to  have  a  high  density  of  metal  and  small  TOI. 

Densely  wooded  environments  are  challenging  for  positioning  the  TEMTADS  sensor.  The 
advantages  of  using  the  TEMTADS  as  a  dynamic  detector  rely  upon  precise  positioning  that 
cannot  be  achieved  using  fiducial  marks  and  interpolation.  Although  RTS  systems  can  be  used  in 
wooded  environments  where  the  line  of  sight  along  the  survey  transect  is  disrupted,  localized  gap- 
fill  surveying  requiring  multiple  base  station  setups  is  needed  to  achieve  100  percent  coverage 
along  the  transects. 
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3.0  PERFORMANCE  OBJECTIVES 


Performance  objectives  for  the  demonstration  were  designed  to  provide  a  basis  for  evaluating  the 
performance  of  the  TEMTADS  with  various  challenging  site  conditions.  The  minimum  acceptable 
criteria  were  the  thresholds  used  to  determine  that  the  system  was  working  properly.  Values  below 
this  threshold  are  a  potential  cause  for  rejection  of  the  data  and  require  a  root  cause 
analysis/corrective  action  if  appropriate  (including  potential  re-collection  of  data).  The  nominal 
success  criteria  represent  the  expected  achievable  threshold.  Failure  to  meet  these  thresholds 
requires  a  discussion  in  the  project  report  but  are  not  cause  for  rejection  of  the  data. 

To  avoid  repetition,  the  specific  performance  objectives  for  the  demonstration  are  presented  along 
with  the  results  in  Section  6  for  NBAFS  and  Section  7  for  TOAR. 


11 


Page  Intentionally  Left  Blank 


12 


4.0  SITE  DESCRIPTION 


This  section  provides  a  brief  summary  of  the  demonstration  sites  at  NBAFS  and  TOAR.  All  site 
information  that  has  immediate  bearing  on  the  performance  of  the  TEMTADS  is  discussed  by 
demonstration  area  below. 

4.1  NBAFS 

The  NBAFS  demonstration  site  is  located  within  MU705,  the  Shooting  Fields,  which  is  a  1 15-acre 
area  located  in  the  northwestern  portion  of  NBAFS,  New  Hampshire  (NH).  Outside  of  the  active 
Operations  Area  the  site  is  used  for  a  variety  of  recreational  activities. 

4.1.1  Site  Location 

The  NBAFS  is  a  2,826-acre  site  located  within  the  towns  of  New  Boston,  Amherst,  and  Mont 
Vernon,  NH.  Outside  of  the  active  Operations  Area,  NBAFS  consists  of  forests,  fields,  hills, 
wetlands,  and  ponds  that  provide  a  variety  of  camping  and  recreational  opportunities.  These 
opportunities  are  restricted  to  NBAFS  personnel  and  guests.  The  Joe  English  Pond  was  once  the 
focal  point  of  recreational  activities  at  NBAFS.  However,  Joe  English  Pond  was  closed  to  all 
recreational  activities  in  1998  because  of  the  probable  presence  of  MEC.  The  land  immediately 
surrounding  NBAFS  is  heavily  wooded,  rural,  and  consists  primarily  of  low-density  residential 
areas.  There  are  several  active  farms  in  the  area  surrounding  NBAFS,  most  of  them  situated  on 
land  adjacent  to  Chestnut  Hill  Road.  The  population  and  housing  growth  in  the  three  surrounding 
communities  is  apparent  from  recent  housing  construction  on  the  land  surrounding  NBAFS.  The 
Joe  English  Conservation  Area,  Amherst,  NH,  is  located  immediately  southeast  of  NBAFS. 

NBAFS  was  nominated  to  the  National  Register  of  Historic  Places  in  2004  as  the  ‘New  Boston 
Air  Force  Station  Archaeological  District’.  When  it  was  nominated,  the  district  included  over 
50  contributing  properties  representing  thousands  of  years  of  human  use.  The  district  includes 
seven  pre-contact  sites  of  unknown  cultural  affiliation,  including  three  small  habitation  areas  and 
four  isolated  finds,  41  historic  properties  (ca.  1780-1940),  and  eight  properties  associated  with  the 
WWII-  and  Cold  War-era  use  of  the  location  by  the  U.S.  military  (ca.  1942-1956).  The  contributing 
properties  include  a  wide  range  of  resource  types  associated  with  cultural  and  social  history,  land 
use,  and  architecture  important  to  the  local,  regional,  state,  and  national  history. 

The  ESTCP  study  was  conducted  on  a  subset  of  the  MU705,  the  Shooting  Fields,  which  is  a 
115-acre  area  located  in  the  northwestern  portion  of  NBAFS,  directly  southeast  of  Joe  English 
Hill.  The  boundaries  of  MU705  were  defined  based  largely  on  the  1995  UXO  clearance  boundary 
data  obtained  from  NBAFS  Geographic  Information  System.  MU 705  is  a  moderately  sloped  area 
with  portions  heavily  forested  with  dense  brush.  Figure  4-1  provides  a  location  map  for  the 
NBAFS  demonstration  site.  Photographs  from  the  site  are  provided  as  Figure  4-2. 
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Figure  4-1.  NBAFS  Location  Map 


Figure  4-2.  NBAFS  Site  Photographs 
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The  NBAFS  site  was  chosen  by  ESTCP  as  one  in  a  series  of  sites  for  demonstration  of  the 
classification  process.  Sites  including  this  one  provide  opportunities  to  demonstrate  the  capabilities 
and  limitations  of  the  classification  process  on  a  variety  of  site  conditions.  This  site  presented  the 
opportunity  to  demonstrate  performance  against  20-mm  projectiles  and  high  anomaly  densities. 
The  general  site  setup  and  grid  system  where  dynamic  TEMTADS  surveys  were  performed  at  the 
site  are  shown  on  Figure  4-3.  Approximately  6.1  acres  were  mapped  using  the  sensor  in  dynamic 
mode.  Cued  investigation  was  subsequently  performed  on  a  subset  area  of  the  site  because  the  high 
density  of  geophysical  anomalies  detected  in  the  subsurface  far  exceeded  the  budgeted  scope  for 
cued  investigations  (1,500  anomalies). 


TEMTADS  Demonstration  Area  at  MU507 

New  Boston  Air  Force  Station,  NH 
ESTCP  Project  Number  MR-201314 
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Figure  4-3.  NBAFS  MU507  Selected  Demonstration  Grids 
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4.1.2  Brief  Site  History 

In  the  fall  of  1941,  the  Federal  Government  acquired  the  2,826  acres  comprising  the  current 
configuration  of  NBAFS.  This  land  was  used  as  an  active  bombing  range  in  support  of  Grenier 
Field  at  nearby  Manchester,  NH,  until  1956.  On  1  October  1959,  the  6594th  Instrumentation 
Squadron  was  activated  at  NBAFS.  The  squadron  was  assigned  to  the  6594th  Aerospace  Test  Wing 
at  Sunnyvale,  California,  and  was  a  tenant  of  the  2235th  Air  Base  Group,  Grenier  Field,  where 
administrative  and  support  facilities  were  maintained. 

Satellite  support  operations  began  on  1  April  1960,  using  van-mounted  equipment  while 
permanent  buildings  were  being  constructed.  By  the  summer  of  1964,  dual-satellite  tracking, 
telemetry,  and  command  capabilities  were  operating  in  permanent  facilities  at  NBAFS.  In  March 
1972,  it  was  announced  that  Grenier  Field  would  close  in  September  of  that  year.  All  support 
facilities  including  supply,  transportation,  fire  protection,  and  civil  engineering  were  moved  to 
NBAFS. 

On  1  October  1979,  the  6594th  Instrumentation  Squadron  was  re-designated  as  Detachment  2,  Air 
Force  Satellite  Control  Facility,  Air  Force  Systems  Command.  On  1  October  1987,  Detachment  2, 
Air  Force  Satellite  Control  Facility  was  re-designated  as  Detachment  2,  2nd  Satellite  Tracking 
Group  and  ownership  was  transferred  from  Air  Force  Systems  Command  to  Air  Force  Space 
Command.  On  1  November  1991,  Detachment  2,  2nd  Satellite  Tracking  Group  was  re-designated 
as  the  23rd  Space  Operations  Squadron.  NBAFS  currently  provides  launch,  operation,  and  on  orbit 
support  for  more  than  170  Department  of  Defense  satellites.  In  addition,  NBAFS  supports  NASA 
missions  as  well  as  NATO  and  other  allied  nation  satellite  operations. 

NBAFS  was  used  as  an  active  bombing  range  in  support  of  Grenier  Field  at  nearby  Manchester, 
NH,  from  the  fall  of  1941  until  1956.  In  addition  to  bombing  activities,  training  and  maneuver 
activities  were  performed  on  the  property  from  1956  until  2002,  when  the  range  officially  closed. 

The  ESTCP  study  area,  MU705,  was  one  of  the  primary  bombing/aerial  targets  used  at  NBAFS 
from  1942  to  1956.  Unserviceable  tanks,  trucks,  and  half-tracks  were  used  as  strafing  targets  for 
machine  guns,  20-mm  cannons,  and  rockets. 

4.1.3  Site  Geology 

The  site  geology  at  NBAFS  had  no  practical  impact  on  the  technology  demonstration. 

4.1.4  Munitions  Contamination 

Munitions  suspected  to  be  present  at  MU705  include: 

•  20-mm  Projectile,  Target  Practice 

•  Practice  Rockets,  2.25-inch  and  5-inch 

•  High  explosive  (HE)  Rockets,  5-inch 

•  Practice  Bombs,  3-pound  (lb),  4.5-lb,  100-lb,  500-lb,  and  1,000-lb 

•  General  Purpose  HE  Bomb,  100-lb 

•  HE  Depth  Bomb,  325-lb  and  350-lb 


16 


•  M69  Incendiary  Bomb 

•  Photoflash  Bomb,  M46 

4.2  TOAR 

The  TOAR  demonstration  site  is  located  within  MRS-R04A  (West)  at  TOAR  FUDS,  which  is 
situated  within  Pennsylvania  State  Game  Lands.  The  site  is  used  for  recreational  activities  such  as 
camping,  hiking,  fishing,  mountain  biking,  and  snowmobiling.  Parts  of  the  MRS  are  located  within 
a  designated  natural  area  open  only  to  passive  recreation  and  hunting. 

4.2.1  Site  Location 

MRS-R04A  (West)  encompasses  approximately  250  acres  and  is  characterized  by  densely 
wooded,  uneven  terrain.  Figure  4-4  shows  photos  of  the  existing  site  conditions  taken  by  CH2M 
during  a  site  reconnaissance  visit  on  19  and  20  May  2015.  Figure  4-5  shows  the  MRS  with  the 
operational  grid  system  used  by  the  munitions  response  (MR)  contractor  performing  work  at  the 
site.  Each  grid  square  measures  100  feet  by  100  feet.  This  figure  also  presents  an  enlarged  view  of 
the  1 1-acre  portion  of  the  MRS  that  had  not  yet  been  cleared  by  the  MR  contractor  and  from  within 
which  up  to  2  acres  was  selected  for  the  demonstration.  Based  on  historical  live-fire  training 
conducted  on  artillery  ranges  at  TOAR  FUDS,  and  the  results  of  a  previously  completed  remedial 
investigation,  this  MRS  encompasses  an  impact  area. 


Pennsylvania 


C=1 


leqeno 

ESTCP  Demonstration 
Boundary  (11  acres) 

I - -  R04A  West  Operational  Grid 

1 - 1  System 


Proposed  ESTCP  Demonstration  Area 
Tobyhanna  FUOS  R04A  West 

ch2m- 


Figure  4-4.  TOAR  MRS-R04A  (West)  ESTCP  Demonstration  Area 
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Figure  4-5.  TOAR  Site  Condition  Photographs 

Information  that  would  enable  estimation  of  anomaly  densities  for  the  MRS  had  not  been  provided; 
therefore,  CH2M  performed  a  digital  geophysical  mapping  survey  with  a  Geometries  G-858G 
cesium  vapor  gradiometer  along  transects  extending  through  the  accessible  portions  of  the 
approximately  11 -acre  area  shown  on  Figure  4-4.  The  intended  nominal  transect  spacing  was 
6  meters  (m);  however,  because  no  vegetation  reduction  was  performed  in  advance  of  the  G-858G 
survey,  transect  spacing  and  percent  coverage  was  variable  across  the  area.  The  objective  of  the 
survey  was  to  assess  relative  anomaly  densities  in  order  to  identify  candidate  grids  for  the 
TEMTADS  and  MPV  demonstrations.  Positional  data  for  the  G-858G  survey  were  recorded  using 
a  Trimble  ProXRS  differential  GPS  system  with  an  intended  sub-meter  horizontal  accuracy.  A 
Geometries  G-856  proton  precession  magnetometer  was  used  as  a  stationary  base  station  to  record 
ambient  magnetic  field  fluctuations  throughout  each  day  of  G-858G  data  collection  in  order  to 
facilitate  evaluation  of  total  field  data  recorded  by  each  G-858G  sensor. 

Processing  and  target  selection  of  the  G-858G  transect  data  were  performed  using  the  Geosoft 
Oasis  montaj  (Geosoft)  software  platform.  Visual  Sample  Plan  (Battelle  Memorial  Institute,  2015) 
was  used  to  obtain  estimates  of  anomaly  density  within  the  11-acre  area.  Transect  paths  and 
calculated  anomaly  densities  are  shown  on  Figure  4-6.  From  these  data,  two  grid  pairs  (78/46- 
79/46  and  82/47-83/47)  were  selected  for  the  demonstration.  CH2M  identified  these  two  sets  of 
grids  because  they  were  sufficiently  far  apart  from  each  other  to  facilitate  concurrent  data 
collection  with  the  TEMTADS  and  MPV  without  the  risk  of  the  two  sensors  interfering  with  each 
other. 
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Figure  4-6.  TOAR  MRS-R04A  (West)  Anomaly  Density  Estimates  and  Selected 

Demonstration  Grids 


4.2.2  Brief  Site  History 

TOAR  was  established  as  Camp  Sumerall  when  the  United  States  purchased  33  square  miles  of 
property  in  Monroe  County  in  1909.  The  facility  was  used  for  a  variety  of  missions  throughout  the 
years. 

The  site  was  first  used  for  machine  gun  and  artillery  training  during  1913.  The  Army  and  National 
Guard  used  the  facility  from  1913  until  1949  for  field  artillery  practice.  Camp  Sumerall  was  also 
used  as  a  training  area  for  tanks  from  July  through  October  1918.  The  ranges  were  the  only  areas 
in  Pennsylvania  where  live  cannon  fire  was  permitted  from  1919  to  1932.  During  this  timeframe, 
the  rounds  were  mainly  75-mm  French  artillery.  The  range  area  between  Warnertown  and  Route 
611  became  a  temporary  Headquarters  Explosives  Depot.  An  estimated  4  million  pounds  of  high 
explosives  were  stored  from  February  1919  through  October  1919.  Bunkers  were  constructed  in 
the  current  State  Game  Lands  127.  The  storage  designation  only  lasted  10  months. 
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4.2.3  Site  Geology 

The  site  geology  at  TOAR  had  no  practical  impact  on  the  technology  demonstration. 

4.2.4  Munitions  Contamination 


Suspected  munitions  within  MRS-R04A  (West)  include  primarily  75-mm  and  155-mm  HE  and 
shrapnel  projectiles;  however,  during  the  remedial  investigation  conducted  within  MRS-R04A 
(West),  37-mm  HE  projectiles  were  reportedly  recovered  along  with  75-mm  and  155-mm  HE 
projectiles. 
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5.0  TEST  DESIGN 


This  section  provides  an  overview  of  the  system  design  and  testing  conducted  during  the 
demonstrations.  Sections  5.1  through  5.5  discuss  the  specific  test  design  elements  for  both 
demonstrations,  Section  5.6  provides  the  details  for  data  collection  at  NBAFS  and  Section  5.7  for 
TOAR. 

5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  broad  objective  of  this  program  was  to  demonstrate  the  methodology  for  the  use  of 
classification  in  the  MR  process.  For  both  demonstrations,  the  three  key  components  of  this 
methodology  were: 

•  Dynamic-mode  survey  for  target  detection  and  development  of  a  cued  target  list 

•  Static-mode  survey  for  cued  target  interrogation  (data  collected  for  extraction  of  target 
parameters  such  as  size,  shape,  and  materials  properties)  and  construction  of  a  ranked 
anomaly  list 

•  Post-acquisition  analysis 

Each  of  the  above  components  was  handled  separately  in  this  program. 

Dynamic  detection  data  were  processed  and  selected  anomalies  were  subsequently  investigated 
through  cued  interrogation.  Individual  cued  data  sets  were  processed  using  existing  routines  in 
UX-  Analyze  to  extract  target  parameters. 

5.2  SITE  PREPARATION 

At  NBAFS  site  preparation  included  vegetation  reduction  (mowing)  of  the  survey  area  before  the 
commencement  of  field  activities.  A  surface  clearance  of  the  demonstration  area  was  conducted 
as  part  of  the  Remedial  Investigation  completed  in  2012. 

At  TOAR,  vegetation  reduction  was  performed  in  the  survey  area  grids  where  a  brush  reduction 
team  removed  vegetation  smaller  than  6  inches  in  diameter.  Low  branches  were  cleared  to  8  feet 
above  ground  to  reduce  obstruction  of  the  RTS  prism;  large  logs  and  fallen  timber  were  not 
removed.  CH2M  also  performed  a  surface  clearance  of  the  selected  demonstration  grids  before 
collection  of  dynamic  data. 

5.3  SYSTEM  SPECIFICATION 

To  avoid  repetition,  see  Section  2  for  a  detailed  description  of  the  overall  system  including  the 
TEMTADS  sensor,  platform,  positioning  systems,  and  data  acquisition  system. 

5.4  SITE  GEODETIC  CONTROL  INFORMATION 

At  NBAFS,  two  first-order  survey  monuments  were  established  at  the  site  by  ESTCP  before 
CH2M  operations  and  were  used  for  all  positioning  during  the  project.  The  monument’s  labels  and 
coordinates  are  provided  in  Table  5-1. 
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Table  5-1.  NBAFS  Geodetic  Control  Locations 


ID 

Easting  (m) 
NAD83/UTM19N 

Northing  (m) 
NAD83/UTM19N 

Elevation 
NAVD88  (m) 

ESTCP1 

283653.23 

4757474.53 

223.880 

ESTCP2 

283617.32 

4757723.70 

217.755 

NAD83  =  North  American  Datum  of  1983 
NADVD88  =  North  American  Vertical  Datum  of  1988 
UTM  =  Universal  Transverse  Mercator 


At  TOAR,  two  control  monuments  were  used  by  the  professional  land  surveyor  (PLS),  GeoMetrics 
GPS,  Inc.,  to  gain  control  of  the  site:  #7  MON  FP5  and  #31  IRC  FP5  AZ.  New  control  stakes  were 
set  for  the  work  area  grids  known  as  MRS-R04A  West,  grids  78/46-79/46  and  82/47-83/47.  These 
points  were  set  by  establishing  a  pair  of  RTK  GPS  control  points  in  the  open  (no  tree  canopy), 
outside  of  the  grid  areas,  running  a  conventional  survey  traverse  to  the  grids,  then  rough-checking 
the  traverse  point  position  with  GPS  under  the  heavy  tree  canopy.  Because  of  the  heavy  woods 
and  terrain,  control  point  recover  sheets  were  not  produced.  From  these  control  points  16  additional 
control  points  were  established  within  the  four  grids.  Their  labels  and  coordinates  are  provided  in 
Table  5-2. 


Table  5-2.  TOAR  Geodetic  Control  Locations 


ID 

Easting  (m) 

Northing  (m) 

Elevation 

NAD83/UTM18  N 

NAD83/UTM18  N 

NAVD88  (m) 

j  1000 

459500.8 

4558192.63 

557.831 

1001 

459543.96 

4558291.43 

559.57 

1002 

459576.81 

4558229.49 

557.356 

1003 

459631.14 

4558194.27 

554.828 

1004 

459736.93 

4558215.96 

557.885 

1010 

459612.62 

4558212.64 

555.64 

1011 

459642.42 

4558175.84 

554.351 

1012 

459668.17 

4558209.79 

555.821 

1013 

459637.54 

4558211.29 

555.246 

1014 

459731.49 

4558208.47 

558.007 

1015 

459750.17 

4558244.65 

557.887 

1016 

459791.55 

4558239.09 

560.63 

1017 

459779.14 

4558208.38 

560.294 

1018 

459728.9 

4558227 

557.984 

1019 

459609.2 

4558176 

554.539 

1  1020 

459731.4 

4558241 

557.627 

1021 

459761.3 

4558225 

558.549 

1022 

459760.8 

4558209 

558.837 
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5.5  SEEDING 


At  NBAFS,  CH2M  placed  110  seeds  within  the  demonstration  site  area  in  accordance  with  the 
parameters  laid  out  in  the  ESTCP  Live  Site  Demonstration  Seeding  Plan  (ESTCP,  2013).  All  seed 
locations  were  blind  to  data  collection  and  analysis  personnel.  Each  flagged  location  was  swept 
with  a  Schonstedt  analog  magnetometer  to  ensure  anomaly  avoidance  (for  safety)  and  a  clean  area 
for  emplacement.  A  hole  was  dug  using  a  ‘plug’  technique  (removing  a  ‘plug’  of  topsoil  for 
replacement  upon  filling  the  hole)  to  hide  any  visual  indications  of  seed  placement.  Seeds  were 
placed  at  the  locations  and  depths  provided  by  the  ESTCP  Program  office.  Physical  characteristics 
of  each  seed  were  recorded  onto  a  whiteboard  and  placed  alongside  the  excavated  hole  and 
photographed  and  the  PLS  recorded  the  locations  of  the  seeds  using  a  RTK  GPS.  A  comprehensive 
list  of  seed  locations  and  burial  information,  including  associated  photographs,  was  provided  to 
the  ESTCP  Program  Office. 

At  TOAR,  after  vegetation  reduction  and  before  conducting  the  dynamic  TEMTADS  and  MPV 
surveys,  CH2M  seeded  the  selected  demonstration  grids.  CH2M  UXO  personnel  practicing 
anomaly  avoidance  buried  20  seed  items  consisting  of  industry  standard  objects  (ISOs).  ISOs 
are  commonly  available  pipe  sections  that  have  been  well-characterized  through  data 
measurements  and  modeling.  A  combination  of  small  schedule  80  ISOs  (diameter  =  1.315 
inches,  length  =  4  inches)  and  medium  schedule  40  ISOs  (diameter  =  2.375  inches,  length  =  8 
inches)  were  used  and  buried  between  1  cm  and  17  cm.  The  PLS  recorded  the  locations  of  the 
blind  seeds  at  the  time  of  emplacement.  All  seeds  locations  were  blind  to  data  collection  and 
analysis  personnel. 

5.6  INSTRUMENT  VERIFICATION  STRIP  AND  TRAINING  PIT 

An  IVS  was  established  at  both  demonstration  sites  in  a  geophysically  quiet  location  proximal  to 
the  survey  area.  The  IVS  was  used  for  pre-survey  and  daily  verification  of  proper  sensor  operation 
and  functionality.  A  schematic  of  the  IVS  at  NBAFS  is  shown  as  Figure  5-1  and  details  of  seed 
items  placed  in  the  IVS  are  listed  in  Table  5-3.  A  schematic  of  the  IVS  at  TOAR  is  shown  as 
Figure  5-2  and  details  of  seed  items  placed  in  the  IVS  are  listed  in  Table  5-4. 
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Instrument  Verification  Strip 

New  Boston  Air  Force  Station,  NH 
ESTCP  Project  Number  MR-201314 


Figure  5-1.  NBAFS  Schematic  of  IVS 
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Table  5-3.  NBAFS  IVS  Construction  Details 


Item 

ID 

Description 

Easting 

(m) 

Northing 

(m) 

Depth 

(cm) 

Inclination 

Azimuth 
(“clockwise 
from  N) 

T-001 

M55A3B1  20-mm 

283568.673 

4757864.312 

2 

Horizontal 

Across  Track 

T-002 

Blank  Space 

283568.46 

4757867.286 

N/A 

N/A 

N/A 

T-003 

2.25-inch  Rocket 

283568.197 

4757870.292 

25 

Horizontal 

Across  Track 

T-004 

Small  ISO80 

283567.771 

4757875.322 

10 

Horizontal 

Across  Track 

Notes: 

Coordinates  provided  in  NAD83,  UTM  Zone  19  North 
N/A  =  not  available 


459760 


Blank 

Line  2  0 


Line  50 


i 

459760 


459764 


ISO-04  ISO-03 

-  x- —  X 


459764 


459768 


459772 


459776 


ISO-02  ISO-01 

-X-  -x4 


-O  Line  2 


OLine  5 


459768 


459772 


459776 


ID 

Easting 

Northing 

Seed  Type 

Depth  (cm) 

Orientation 

ISO-01 

459771.4 

4558208.224 

Small  ISO80 

15 

Horizontal;  cross  track 

ISO-02 

459769.9 

4558208.317 

Small  IS080 

10 

Horizontal;  along  track 

ISO-03 

459767.2 

4558208.351 

Medium  ISO40 

30 

Horizontal;  cross  track 

ISO-04 

459765.3 

4558208.276 

Small  15080 

15 

Vertical 

Instrument  Verification  Strip 
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Figure  5-2.  TOAR  Schematic  of  IVS 
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Table  5-4.  TOAR IVS  Construction  Details 


Item 

ID 

Description 

Easting 

(m) 

Northing 

(m) 

Depth 

(cm) 

Inclination 

Azimuth 

ISO-1 

Small  ISO80 

459771.40 

4558208.22 

15 

Horizontal 

Across  Track 

ISO-2 

Small  ISO80 

459769.86 

4558208.31 

10 

Horizontal 

Along  Track 

Notes: 

Coordinates  provided  in  NAD83,  UTM  Zone  18  North 


At  NBAFS,  a  test  pit  and  test  stand  were  established  adjacent  to  the  IVS.  These  were  used  to  collect 
signatures  of  a  subset  of  TOI  expected  to  be  found  at  the  site.  Items  for  which  static  test  data  were 
collected  are  detailed  in  Table  5-5.  The  data  from  these  measurements  were  added  to  the 
UX-Analyze  (UXA)  library  used  for  the  cued  data  analysis.  No  test  pit  data  were  collected  at 
TOAR. 

Table  5-5.  NBAFS  Test  Stand  Items  and  Orientations 


Item  ID 

Depths 

(cm) 

Orientations 

Small  ISO80 

2,  10,  20,  30 

Horizontal,  Vertical,  45° 

20-mm  ATC  Projectile 

2,5 

Horizontal,  Vertical,  45°  ; 

2.25-inch  Rocket 

2,5 

Horizontal,  Vertical,  45° 

2-inch  x  1/2-inch  Pipe  Nipple  with  Cap  (20-mm  surrogate) 

5,  10,  15 

Horizontal,  Vertical,  45° 

20-mm  Shell  Casing 

5,  10 

Horizontal,  Vertical,  45° 

Ground  Signal  Flare 

5,  10,  15 

Horizontal,  Vertical,  45° 

M102A1  Fuze 

15 

Horizontal,  Vertical,  45° 

M83  Smoke  Grenade 

10,  15 

Horizontal,  Vertical 

M8  Practice  Land  Mine 

10,  15 

Horizontal,  Vertical 

Ml 20  Fuze 

10,  15,  20 

Horizontal,  Vertical 

Ml 03  Fuze 

15,20 

Horizontal,  Vertical 

3-1/2-inch  Rocket  Warhead,  Motor  and  Fuze 

20 

Horizontal 

3-1/2-inch  Rocket  Warhead 

15,20 

Horizontal,  Vertical,  45° 

3-1/2-inch  Rocket  Motor  and  Fuze 

15 

Horizontal,  Vertical,  45° 
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5.7  SENSOR  CALIBRATION 


The  TEMTADS  sensors  used  for  both  demonstrations  were  calibrated  during  the  initial 
commissioning  of  the  system  at  the  NRL  facility  at  Blossom  Point,  Maryland.  The  system 
calibration  was  verified  on  a  regular  basis  via  function  tests  and  daily  IVS  surveys.  The  function 
tests  involve  measurement  of  the  system  response  to  an  ISO  placed  in  a  known  location  relative  to 
the  sensor.  Function  tests  were  performed  a  minimum  of  twice  per  day  to  verify  the  proper 
operation  of  all  of  the  sensor  transmit  and  receive  components.  Results  of  the  function  tests  and 
IVS  surveys  are  provided  in  the  discussion  of  quality  control  (QC)  testing  in  Section  6  and 
Section  7  for  NBAFS  and  TOAR  respectively. 

5.8  DATA  COLLECTION 

For  clear  presentation  of  data  collection  activities,  this  discussion  is  presented  in  two  general 
sections  corresponding  to  the  two  demonstration  areas.  The  summary  of  data  collection  events  at 
NBAFS  is  presented  first  followed  by  TOAR. 

5.8.1  NBAFS 

5.8. 1.1  Dynamic  Data  Collection  (Mappping  Survey ) 

In  the  first  phase  of  the  demonstration,  the  TEMTADS  was  operated  in  dynamic  (mapping)  mode 
in  order  to  generate  a  detection  map  and  target  list.  Mapping  was  performed  over  approximately 

6.1  acres  within  the  survey  area  at  MU705.  The  dynamic  detection  survey  took  place  over  the 
course  of  1 1  survey  days  during  the  time  period  from  18  July  to  14  August  2013  (during  this  time 
period  there  were  a  number  of  no-collection  days  because  of  equipment  failures  as  well  as  weather 
delays  and  non- worked  weekends).  The  positions  of  each  measurement  were  determined  using  a 
RTK  GPS  antenna  mounted  at  the  center  of  the  array,  coil  geometry  relative  to  the  GPS  antenna, 
and  the  platform  attitude  (pitch,  roll,  and  yaw)  derived  from  the  IMU.  Data  were  acquired  along  a 
series  of  parallel  transects  with  a  60  cm  (nominal)  line  spacing.  This  spacing  was  designed  to 
provide  overlap  coverage  of  20  cm  to  reduce  the  chance  of  data  gaps.  Figure  5-3  shows  a  mosaic 
of  the  data  collected  at  the  site. 
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Figure  5-3. 
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5.8. 1.2  Dynamic  Data  Quality  Control 

Throughout  the  course  of  the  dynamic  detection  survey,  the  TEMTADS  system  was  tested  at  the 
IVS  on  a  twice  daily  basis  to  verify  proper  system  operation.  In  order  to  measure  precision  of  the 
system,  ongoing  analysis  was  performed  on  the  IVS  detection  results,  with  each  successive  day’s 
results  compared  to  the  averaged  results  of  all  previous  IVS  surveys  for  detection  offset  and 
amplitude  response  of  each  seed  item. 

The  positions  were  derived  from  the  dynamic  monostatic,  Z-component  response  amplitude  using 
Geosoft’s  automatic  peak  picking  algorithm.  Figure  5-4  presents  the  position  errors  (relative  to 
the  ground  truth)  for  each  of  the  IVS  items.  The  errors  are  within  the  stated  objective  of  0.2  m 
(objectives  are  presented  in  Section  6  for  NBAFS). 


Figure  5-4.  NBAFS  Dynamic  IVS  Survey  Positioning  Results 

The  response  amplitude  values  were  also  derived  using  the  automatic  peak  detection  algorithm 
and  are  presented  in  Figure  5-5.  The  variability  of  these  values  were  within  the  stated  objective  of 
20  percent  root  mean  square  (RMS). 
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In  addition  to  the  daily  IVS  measurements,  the  functionality  of  the  TEMTADS  sensor  was  assessed 
daily  using  a  system  ‘function  test’  whereby  the  system  response  was  challenged  by  placing  an 
ISO,  1.5-inch  x  4-inch,  schedule  80  thickness  pipe  nipple  (small  ISO)  on  the  top  of  the  array 
housing.  The  function  test  results  are  presented  on  Figure  5-6. 


Figure  5-6.  NBAFS  Dynamic  Survey  Sensor  Function  Test  Results 
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These  tests  identified  a  failure  of  the  Rx2X  coil  on  Julian  Day  (JD)  224  (August  12)  and  failures 
of  the  Rx2  cube  (X,  Y,  and  Z  components)  coil  on  JD  226  (August  14).  The  affected  data  were 
noted  in  the  data  delivery  package.  The  decision  was  made  to  not  re-acquire  data  from  these  dates 
because  the  effect  on  the  results  was  determined  to  be  minimal. 

5.8. 1.3  Dynamic  Data  Processing  and  Analysis 

The  raw  instrument  data  files  comprised  of  raw  sensor  data,  RTK  GPS  data,  and  IMU  measurement 
data  converted  to  comma  separated  values  (*.csv)  format  using  NRL’s  ConvertTEMTADS  data 
conversion  application  for  import  into  Geosoft’s  UXA  advanced  analysis  software  environment.  The 
*.csv  file  contains  the  sensor  data  merged  with  the  RTK  GPS  and  IMU  data  providing  georeferenced 
positions  for  each  TEMTADS  transmit/receive  (Tx/Rx)  combination  measurement. 

The  dynamic  detection  was  performed  using  the  amplitude  response  of  the  monostatic,  vertical 
component  measurements  only.  This  approach  is  analogous  to  standard  Geonics  EM61-MK2 
detection  surveys  with  the  main  difference  being  that  the  data  are  much  more  dense  and  precise, 
providing  a  much  higher  resolution  data  set.  Dynamic  detection  data  were  imported  into  the  Geosoft 
UXA  module  for  processing  and  analysis.  Individual  sensor  data  were  assigned  coordinates  based  on 
sensor  offset  and  IMU  data  relative  to  the  center  mounted  RTK  GPS  location.  Positioned  data  were 
then  exported  to  a  located  sensor  database  for  data  levelling  and  target  selection. 

Dynamic  detection  data  were  levelled  by  deriving  a  background  model  (comprised  of  long 
wavelength  signals  because  of  spatially  stable  soil  response  and  sensor  ‘zero  level’  drift)  and 
subtracting  this  model  from  the  raw  data  to  derive  a  levelled  data  set.  Because  of  the  anomaly 
density  encountered,  typical  demedian  filters  used  to  derive  the  background  model  were  not 
appropriate;  as  a  result,  the  model  was  derived  using  a  deminimum  filter  where  the  minimum  over 
a  large  moving  window  is  calculated  and  then  low-pass  filtered  to  remove  high  frequency  artifacts. 
Figure  5-7  shows  example  results  of  both  filters  on  the  raw  data. 


'Demedian'  Background  Model  'Deminimum'  Background  Model 


Figure  5-7.  NBAFS  Dynamic  Survey  Background  Removal 
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The  final,  leveled  data  for  each  grid  were  interpolated  to  grid  nodes  evenly  spaced  at  0.05  m  using 
Geosoft’s  minimum  curvature  gridding  routine.  The  interpolated  Geosoft  grid  (GRD)  files  were 
mosaicked  to  create  a  full  site  GRD  file  from  which  anomaly  selection  was  performed. 

CH2M  provided  raw  and  preprocessed  data  to  the  ESTCP  Program  Office  for  archiving  in  the 
form  of  *.tem  (Input  template  file),  *.csv  and  Geosoft  format  databases  (*.gdb).  In  consultation 
with  the  Program  Office,  CH2M  identified  anomalies  in  the  MU507  dynamic  detection  dataset. 
Response  amplitude  anomalies  were  selected  from  the  full  site  GRD  using  the  Geosoft  Blakely 
grid  peak  detection  algorithm.  The  cut-off  threshold  was  2.3  millivolts  per  ampere  (mV/A).  All 
anomaly  selections  were  manually  reviewed  by  the  processing  geophysicist  and  manual  additions 
or  deletions  were  performed  where  required.  The  threshold  was  validated  by  analyzing  dynamic 
data  collected  over  items  buried  in  the  test  pit  before  the  dynamic  survey  of  the  demonstration 
area.  The  test  pit  items  included  20- mm  projectiles  (the  smallest  TOI  expected  at  the  site)  at  various 
depths  and  orientations  to  determine  relevant  system  response  values.  Site-  and  system- specific 
noise  information  derived  from  sample  survey  data  were  used  to  validate  the  proposed  anomaly 
selection  amplitude  threshold. 

5. 8. 1.4  Cued  Data  Collection  ( Static  Survey ) 

After  collection  and  analysis  of  the  dynamic  TEMTADS  data  were  complete,  the  ESTCP  Program 
Office  selected  a  subset  of  1,500  anomalies  (out  of  a  total  of  18,373  anomalies  detected  by  the 
dynamic  survey)  for  cued  interrogation.  This  down-selection  was  made  to  keep  the  project  within 
its  original  scope  (the  anomaly  density  was  an  order  of  magnitude  higher  that  was  originally 
anticipated).  The  selected  anomalies  were  located  within  Grid  J22  (Figure  5-8).  Most  of  the 
anomalies  were  within  the  region  located  west  of  the  road  and  east  of  the  forested  region  (the  edge 
of  the  survey  area).  A  small  number  of  anomalies  were  located  east  of  the  road  -  these  were 
selected  primarily  to  include  some  blind  seeds  that  otherwise  would  not  be  have  been  investigated. 
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Figure  5-8.  NBAFS  Anomalies  Selected  for  Cued  Interrogation  -  Grid  J22 

Cued  surveys  were  performed  with  the  TEMTADS  over  the  course  of  23  days  from  1  August 
(JD  213)  to  23  August  (JD  235),  2013.  Data  were  recorded  electronically  as  collected  on  the 
TEMTADS  backpack  data  acquisition  computer  hard  drive.  The  collected  data  were  copied  and 
backed  up  daily  onto  removable  media  and  data  were  transferred  daily  to  the  data  analyst  for  QC/ 
analysis. 


33 


5.8. 1.5  Cued  Data  Quality  Control 

The  QC  implemented  throughout  the  cued  data  acquisition  included  the  following: 

•  IVS  measurements 

•  Function  tests 

•  Transmit  current  and  receiver  decay  monitoring 

•  Field  inversion  monitoring 

•  Recollection  where  horizontal  target  location  was  offset  by  more  than  40  cm 

Throughout  the  course  of  the  cued  data  acquisition,  the  TEMTADS  system  was  tested  at  the  IVS 
on  a  twice  daily  basis  to  verify  sensor  functionality.  The  daily  IVS  measurements  were  inverted 
and  the  extrinsic  parameter  (source  position)  and  intrinsic  parameter  (source  betas)  results  were 
monitored  and  recorded.  These  results  are  presented  in  Figure  5-9  and  Figure  5-10,  respectively. 
The  values  obtained  were  within  the  stated  limits,  with  the  exception  of  one  of  the  IVS  positions. 
On  the  final  measurement  of  8  August  (JD  220)  the  GPS  base  station  battery  failed,  resulting  in  an 
inaccurate  position  location  for  the  last  of  the  three  emplaced  items.  Because  this  failure  happened 
after  the  day’s  data  collection,  it  had  no  adverse  implications  for  the  usability  of  the  day’s  data. 
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TEMTADS  2x2  IVS  Response  Test 
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Figure  5-10.  NBAFS  Cued  Data  Dipole  Fit  p  Amplitude  Results 

In  addition  to  the  twice-daily  IVS,  in-field  sensor  functionality  tests  were  performed  throughout 
each  survey  day  to  confirm  that  the  TEMTADS  system  components  were  functioning  within 
project  specifications.  Sensor  function  tests  were  performed  during  each  background  data 
collection  event.  The  sensor  function  test  results  are  shown  on  Figure  5-11.  These  tests  identified 
a  failure  of  the  Rx2X  coil  on  JD  224  and  failures  of  the  Rx2Z  coil  on  JDs  232  and  234-235 
(11,  13,  14  August  2013). 


Figure  5-11.  NBAFS  Cued  Data  Function  Test  Results 
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The  effect  of  the  missing  data  was  tested  and  assumed  to  be  minimal  as  long  as  the  failed  data 
were  rejected  from  the  inversion.  This  is  illustrated  by  Figure  5-12.  This  figure  presents  inversion 
results  for  a  20-mm  target  located  at  5  positions  under  the  sensor  (center  and  under  each  coil)  with 
the  Rx2Z  coil  inoperable.  On  the  top  row  the  bad  data  are  included  in  the  inversion  and  corrupt 
the  inversion  results  as  evidenced  by  the  lower  library  match  values.  On  the  bottom  row  the  Rx2Z 
data  are  excluded  from  the  inversion  and  the  inversion  results  are  accurate.  The  affected  data  were 
noted  in  the  data  delivery  package. 


Figure  5-12.  NBAFS  Effect  of  Missing  Rx2Z  on  Inversion/Classification  Results  for  a 

20-mm  Projectile 

5.8. 1.6  Cued  Data  Processing  and  Post-acquisition  Analysis 

Static  data  points  acquired  with  the  TEMTADS  platform  located  over  each  of  the  targets  identified 

using  the  processes  described  above  were  analyzed  to  extract  target-specific  parameters.  The 

general  processing  flow  is  described  in  the  following  steps: 

1.  Target  Association:  cued  measurements  are  associated  with  a  particular  target  from  the  cued 
target  list 

2.  Feature  Selection:  statistical  analyses  are  performed  to  determine  target  features 

3.  Library  Matching:  target  polarization  curves  extracted  for  each  target  are  matched  to  a  library 
of  munitions  known  to  exist  in  the  area  of  interest 

4.  Analyst  Calibration  Digs:  used  to  determine  the  source  of  any  identified  clusters  of  ‘like 
signatures’  as  well  as  to  inform  the  analyst  with  respect  to  the  placement  of  decision  thresholds 
in  the  prioritized  list 
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5.  Prioritized  Target  List  Assembly:  final  AGC  determinations  on  the  set  of  cued  targets  and 
assembly  of  a  final  prioritized  dig  list 

Cued  data  were  imported  into  the  Geosoft  UXA  module  for  data  QC  and  inversion  modeling.  The 
data  were  levelled  using  background  data  collected  at  frequent  time  intervals  over  a  nearby 
anomaly-free  background  location.  The  measurements  used  for  background  correction  were 
reviewed  for  variability  and  to  identify  any  outliers  which  may  correspond  to  measurements  over 
subsurface  metal.  To  minimize  errors  in  the  background  removal  process,  spatial  and  temporal 
distance  between  the  background  and  target  measurements  were  minimized. 

Parameter  estimation  was  performed  using  the  Geosoft  UXA  module.  Target  data  were  inverted 
using  both  single-source  and  multi-source  dipole  response  models  to  estimate  target  parameters. 
The  principle  parameters  of  interest  for  use  in  classification  of  the  targets  were  the  three 
polarizabilities  (pi,  P2,  and  P3)  estimated  for  each  target  by  UXA.  In  addition  to  estimates  for  the 
three  Ps  for  each  target,  an  estimated  location  and  depth  was  also  returned  by  UXA  for  each  target 
during  inversion. 

CH2M  provided  the  raw  data  (e.g.  *.tem  and  *.gps),  and  digital  as  well  as  analog  field  notes  to  the 
ESTCP  Program  Office  for  archiving. 

5.8. 1.7  Analyst  Calibration  Digs 

Upon  initial  review  of  the  inversion  results.  A  set  of  targets  were  selected  as  analyst  calibration 
digs  (ACD).  These  data  would  be  used  to  determine  the  source  of  any  identified  clusters  of  ‘like 
signatures’  as  well  as  to  inform  the  analyst  with  respect  to  the  placement  of  decision  thresholds  in 
the  prioritized  list. 

After  receipt  of  the  dig  results  for  the  requested  ACD,  the  decision  was  made  to  abandon  attempts 
to  classify  the  results.  More  than  40  percent  of  the  ACD  resulted  in  4  or  more  discrete  objects 
being  found  for  a  given  flag  location  (a  retrospective  analysis  of  all  of  the  dig  results  indicated  that 
33  percent  of  the  digs  resulted  in  4  or  more  discrete  objects). 

Comparison  with  the  ground  truth  revealed  that  the  multi-solver  inversion  routine  was  not  able  to 
consistently  extract  the  correct  signatures  for  the  TOI  at  the  site.  Figure  5-13  presents  an  example 
of  a  false  positive  finding  where  the  derived  signatures  do  not  match  the  recovered  sources, 
resulting  in  a  non-TOI  result  being  classified  as  a  TOI.  Although  some  percentage  of  false 
positives  are  expected,  one  of  the  goals  of  classification  is  to  minimize  these  (and  thus  reduce  the 
number  of  required  intrusive  investigations).  The  fact  that  the  inversion  results  do  not  accurately 
represent  the  actual  subsurface  items  is  demonstrated  by  this  figure. 
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Classification  Example  1: 
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Figure  5-13.  NBAFS  Example  of  False  Positive  Classification  Results 

False  negatives  present  a  much  more  serious  problem.  An  example  of  a  false  negative  is  shown  on 
Figure  5-14  where  two  20-mm  projectiles  were  recovered,  but  the  inversion  results  for  that  flag 
location  did  not  have  a  signature  that  was  a  good  fit  to  these  TOI.  These  signatures  did  fit  to  a 
37-mm  but  with  a  low  fit  metric  that  resulted  in  a  non-TOI  classification. 
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Figure  5-14.  Example  of  'False  Negative'  Classification  Results 
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In  both  of  the  examples  shown,  the  multi-solver  inversion  failed  to  extract  signatures  consistent 
with  the  objects  recovered.  Qualitative  analysis  of  the  results  indicates  that  the  multi-solver  results 
become  unreliable  when  4  or  more  targets  are  below  the  sensor.  It  bears  mention  that  this  was  not 
a  quantitative  assessment  nor  does  this  observation  support  any  generalized  conclusions  regarding 
the  performance  of  the  multi-solver;  this  observation  only  applies  for  the  specific  circumstances 
of  this  demonstration  (sites  with  larger  minimum  TOI  size  might  show  a  different  result).  Given 
that  33  percent  of  the  intrusive  investigations  resulted  in  three  or  more  recovered  items  and 
70  percent  of  the  excavations  resulted  in  at  least  one  TOI  (thus  the  technology  is  assumed  to  be 
unsuitable  at  this  site),  further  investigation  into  the  performance  of  the  multi-solver  was  not 
undertaken. 

5. 8. 1.8  Classification 

After  review  of  the  analysis  calibration  data,  an  approach  that  reliably  derived  accurate  intrinsic 
parameters  when  there  were  more  than  three  sources  under  the  sensor  was  not  able  to  be  found. 
Furthermore,  the  fact  that  70  percent  of  the  intrusive  investigations  resulted  in  TOI  indicated  that 
submission  of  a  final  prioritized  list  would  add  no  value. 

5. 8. 1.9  Intrusive  Activity  and  Procedures 

CH2M  performed  intrusive  operations  from  28  August  through  24  September  2013.  All  prosecuted 
targets  were  investigated  and  documented  according  to  the  procedures  outlined  in  the  Intrusive 
Investigation  Data  Collection  Instructions.  These  included  the  following: 

•  Reacquisition  of  targets.  Targets  selected  for  intrusive  investigation  were  reacquired 
using  a  Trimble  RTK  GPS  and  marked  in  the  field  using  PVC  pin  flags. 

•  Intrusively  investigate  the  anomaly.  Anomalies  were  excavated  to  30  cm  below  the 
expected  depth  below  ground  surface,  within  a  50-cm  halo  from  the  marked  out  anomaly 
location. 

•  Identify  recovered  item.  All  items  recovered  were  inspected  by  the  UXO  Safety  Officer 
and  Senior  UXO  Supervisor  to  ensure  that  each  item  was  properly  identified  and  properly 
documented. 

•  Munitions  Response  Site  Information  Management  System  (MRSIMS)  Data  Entry/ 
Whiteboard  and  photo.  Field  observations  of  each  recovered  item  were  entered  into 
CH2M’s  MRSIMS  field  tablets.  The  exact  location  and  depth  of  each  item  was  recorded 
using  a  Trimble  RTK  GPS.  Required  information  was  written  onto  a  whiteboard  and  a 
photo  was  taken  with  the  item. 

•  Bag  and  label  item.  All  recovered  items  were  placed  in  zip-lock  bags  and  labeled. 

•  Post-dig  clearance.  Before  declaring  a  dig  complete,  each  area  was  swept  with  a 
Schonstedt  magnetometer  to  determine  if  any  additional  items  remained 

•  Backfill  hole.  Once  the  excavation  was  declared  clear,  the  hole  was  backfilled  to  grade. 

As  NBAFS  is  included  in  the  National  Register  of  Historic  Places,  all  intrusive  investigations 
were  carried  out  in  compliance  with  the  Memorandum  of  Agreement  between  the  U.S. 
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Air  Force  and  the  New  Hampshire  Historic  Preservation  Office  (U.S.  Air  Force,  2013). 
Archaeological  monitoring  was  performed  by  a  certified  archaeologist  from  CH2M’s  Archaeology 
subcontractor,  Landmark  Archaeology,  Inc.,  and  any  artifacts  of  significance  that  were  recovered 
during  intrusive  investigations  were  properly  documented  and  a  report  provided  to  the  New 
Hampshire  Division  of  Historic  Resources  (Landmark  Archaeology,  Inc.,  2014). 

5.8.1.10  Deliverables 

The  following  deliverables  resulted  from  the  data  collection  at  NBAFS: 

1.  Dynamic  Detection  Data:  Raw  and  processed  dynamic  detection  data  were  provided  to 
the  ESTCP  program  office,  along  with  a  final  target  list  based  on  the  established  detection 
threshold. 

2.  Cued  Data:  Raw  sensor  data  (*.tem)  and  associated  GPS/IMU  data  (*.gps) 

3.  Cross-Reference  List:  A  text  readable  table  that  associates  TEMTADS  filenames  with 
each  Target  ID  and  provides  any  applicable  collection  notes. 

5.8.2  TOAR 

5.8.2. 1  Dynamic  Data  Collection  (Mappping  Survey) 

In  the  first  phase  of  the  demonstration,  the  TEMTADS  was  operated  in  dynamic  (mapping)  mode 
in  order  to  generate  a  detection  map  and  target  list.  Dynamic  detection  surveys  were  performed 
over  the  course  of  11  survey  days,  from  12  August  to  4  September  2015  (during  this  period  there 
were  a  number  of  no-collection  days  because  of  equipment  failures  as  well  as  weather  delays  and 
non- worked  weekends).  There  were  a  total  of  8  days  of  initial  data  collection  and  an  additional  3 
days  of  effort  filling  gaps  in  coverage.  The  position  of  each  measurement  was  determined  using  a 
RTS  prism  mounted  at  the  center  of  the  array,  coil  geometry  relative  to  the  RTS  prism,  and  the 
platform  attitude  (pitch,  roll,  and  yaw)  derived  from  the  IMU.  CH2M’s  field  team  used  ropes  and 
flagging  to  perform  data  collection  at  a  line  spacing  of  50  cm,  which  provided  an  overlap  coverage 
of  30  cm  to  reduce  the  chance  of  data  gaps.  Figure  5-15  and  Figure  5-16  show  mosaics  of  the  data 
collected  at  78/46  and  79/46  (the  western)  and  82/47  and  83/47  (the  eastern)  grid  sites, 
respectively. 

Dynamic  data  collection  was  hindered  by  the  rough  terrain  (boulders,  swamp,  felled  trees,  and 
standing  trees)  and  the  need  to  reestablish  position  control  on  a  separate  control  point  once  the 
angle  from  the  RTS  ‘gun’  became  sub-optimal  or  a  tree  blocked  too  much  of  the  ‘gun’  site. 

Because  of  site  access  time  constraints,  it  was  decided  with  input  and  concurrence  from  the  ESTCP 
Program  Office  to  focus  on  100  percent  completion  of  a  single  grid.  Grid  82/47  (Figure  5-16)  had 
gaps  identified  by  CH2M  and  the  data  were  collected  until  only  physical  obstacles  (not  RTS  ‘gun’ 
shadows)  inhibited  data  collection.  Because  it  took  an  additional  3  days  to  complete  the  gap  fill  on 
one  grid  (one-quarter  of  the  planned  survey  area),  by  extrapolation  it  is  estimated  that  an  additional 
9  days  would  have  been  required  to  fill  all  of  the  gaps  in  the  planned  survey  areas. 

Of  the  0.89-acre  footprint  cleared,  approximately  0.71  acre  of  dynamic  detection  data  were 
collected  with  the  TEMTADS.  CH2M  evaluated  the  dynamic  TEMTADS  data  and  selected 
429  targets  based  on  anomaly  selection  thresholds  derived  from  IVS  and  dynamic  test  data. 
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Figure  5-15.  TOAR  Dynamic  TEMTADs  2x2  Survey  in  Grids  78/46  and  79/46  (Western) 
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Figure  5-16.  TOAR  Dynamic  TEMTADS  2x2  Survey  in  Grids  82/47  and  83/47  (Eastern) 
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5.8. 2.2  Dynamic  Data  Quality  Control 

Throughout  the  course  of  the  dynamic  detection  survey,  the  TEMTADS  system  was  tested  at  the 
IVS  on  a  twice-daily  basis  to  verify  proper  system  operation.  In  order  to  measure  precision  of  the 
system,  ongoing  analysis  was  performed  on  the  IVS  detection  results,  with  each  successive  day’s 
results  compared  to  the  averaged  results  of  all  previous  IVS  surveys  for  detection  offset  and 
amplitude  response  of  each  seed  item. 

The  positions  were  derived  from  the  dynamic  monostatic,  Z-component  response  amplitude 
anomaly  peaks  using  Geosoft’s  automatic  peak  picking  algorithm.  Figure  5-17  (left)  presents  the 
position  offsets  (relative  to  the  ground  truth)  for  each  of  the  IVS  items.  Figure  5-17  (right)  presents 
the  position  offsets  from  average  position.  The  errors  were  outside  the  stated  objective  of  0.25  m 
(TOAR  objectives  presented  in  Section  7  of  this  document). 
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Figure  5-17.  TOAR  Dynamic  TEMTADS  2x2  IVS  Survey  Position  Results  (Initial 

Processing) 

During  the  field  operations,  the  lack  of  positioning  accuracy  was  assumed  to  be  a  function  of  the 
RTS  operating  in  the  wooded  environment.  After  the  project  was  completed,  a  review  of  the 
processing  approach  revealed  that  the  data  processor  had  failed  to  account  for  the  extended  height 
of  the  RTS  prism  above  the  TEMTADS  array.  Because  of  this,  the  pitch  and  roll  positioning 
corrections  were  not  accurate,  and  the  accuracy  of  the  results  was  compromised.  The  data  were 
reprocessed  using  the  correct  positioning  approach,  and  the  results  are  provided  in  Figure  5-18. 
While  these  results  indicate  considerable  improvement,  there  remains  very  little  difference 
between  the  accuracy  (results  against  ground  truth)  and  the  precision  (variability  in  the  results).  This 
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indicates  that  the  positioning  methodology  used  does  not  have  the  same  precision  as  a  typical  GPS- 
enabled  approach.  However,  the  precision  in  the  results  is  comparable  to  that  of  a  GPS-enabled 
EM61-MK2  survey,  and  the  dig  radius  of  1  m  used  during  the  intrusive  investigation  was  sufficient 
to  mitigate  the  effects  of  these  increased  errors  on  locating  the  anomaly  sources. 


Figure  5-18.  TOAR  Dynamic  TEMTADS  2x2  IVS  Survey  Position  Results  (Reprocessing) 

In  addition  to  the  daily  IVS  measurements,  the  functionality  of  the  TEMTADS  sensor  was  assessed 
daily  using  a  system  ‘function  test’  whereby  the  system  response  was  challenged  by  placing  a 
small  ISO  (schedule  80)  on  the  top  of  the  array  housing.  The  function  test  results  are  presented  on 

Figure  5-19. 
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5.8.23  Dynamic  Data  Processing  and  Analysis 

The  raw  data  files,  comprising  raw  sensor,  RTS,  and  IMU  data,  were  loaded  into  Geosoft’s  UXA 
software  environment.  The  RTS  and  IMU  data  were  merged  with  the  sensor  data  to  provide 
georeferenced  positions  for  each  TEMTADS  Tx/Rx  combination  measurement.  The  monostatic  Z- 
component  Rx  coil  measurements  were  used  as  the  basis  for  amplitude  response  anomaly  detection. 

The  background  monostatic  Z-component  responses  were  removed  using  a  de-median  filter  in 
which  long  wavelength  signals  (because  of  spatially  stable  soil  response  and  sensor  ‘zero  level’ 
drift)  were  modeled  and  removed  by  calculating  the  median  over  a  large  moving  window  and 
subtracting  the  median  from  the  unleveled  data.  The  final  (leveled)  data  for  each  grid  were 
interpolated  to  grid  nodes  evenly  spaced  at  5  cm  using  Geosoft’s  minimum  curvature  gridding 
routine.  The  interpolated  Geosoft  grid  files  were  mosaicked  to  create  a  full-site  Geosoft  grid  file 
from  which  anomalies  were  selected. 

Targets  were  selected  using  amplitude  response  detection  by  applying  Geosoft’s  peak  detection 
function  to  the  interpolated  monostatic,  Z-component  responses  at  time  gate  5  (0.137  ms).  This  is 
similar  to  conventional  EM61-MK2  target  detection,  with  the  only  difference  being  the  higher 
resolution  of  the  TEMTADS  sensor  because  of  the  smaller  Tx/Rx  coil  footprints  and  more  densely 
sampled  data  set. 

Detection  performance  is  a  function  of  the  signal-to-noise  ratio  (SNR)  of  the  detection  method, 
where  signal  is  derived  as  the  peak  anomaly  response  and  the  noise  is  calculated  as  the  root  mean 
square  of  the  non-anomalous  responses.  Typically,  a  SNR  of  5  is  used  to  maximize  detection  of 
real  targets  while  minimizing  false  detections  because  of  noise  in  the  data.  The  site-specific  noise 
levels  were  estimated  by  calculating  the  standard  deviation  of  the  leveled  data  (assuming  perfect 
leveling,  root  mean  square  noise  is  equivalent  to  the  standard  deviation  of  the  signal)  in  non- 
anomalous  regions.  Noise  levels  of  0.46  mV/A  resulted  in  a  detection  threshold  of  2.3  mV/A. 
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This  threshold  provides  a  maximum  reliable  detection  depth  (assuming  worst  case  orientation)  of 
30  cm  for  small  ISOs  (small  ISOs  are  similar  in  response  to  37-mm  projectiles).  Note  that 
optimally  oriented  small  ISOs  and  37-mm  projectiles  will  be  detected  at  greater  depths.  Using  this 
threshold,  anomalies  were  automatically  selected  from  the  gridded  data  using  Geosoft’s  peak 
detection  algorithm.  All  anomaly  selections  were  manually  reviewed  by  the  processing 
geophysicist,  and  manual  additions  or  deletions  were  performed  where  required. 

CH2M  selected  a  total  of  429  anomaly  locations  for  cued  interrogation.  There  were  165  selections 
in  the  western  grids  (Figure  5-20)  and  264  selections  in  the  eastern  grids  (Figure  5-21).  In  addition 
to  these,  68  MPV  targets  selected  by  Black  Tusk  Geophysics  were  added  to  the  cued  interrogation 
and  intrusive  investigation  lists.  These  targets  were  MPV  targets  located  in  the  100  percent 
coverage  area  (eastern  grids)  that  did  not  have  a  corresponding  TEMTADS  anomaly  location  (due 
primarily  to  the  fact  the  MPV  could  access  locations  that  the  TEMTADS  could  not). 

5.8.2.4  Cued  Data  Collection 

Cued  surveys  were  performed  with  the  TEMTADS  over  the  course  of  7  days,  from  4  September 
(JD  247)  to  17  September  (JD  260),  2015.  Data  were  recorded  electronically  as  collected  on  the 
TEMTADS  backpack  data  acquisition  computer  hard  drive.  The  collected  data  were  copied  and 
backed  up  daily  onto  removable  media  and  transferred  daily  to  the  data  analyst  for  QC  analysis. 

All  of  the  429  targets  detected  by  the  TEMTADS  were  reacquired  for  cued  interrogation  with  the 
TEMTADS.  Cued  measurements  at  the  68  MPV  target  locations  that  were  not  selected  by  the 
TEMTADS  were  also  collected,  resulting  in  a  total  of  497  reacquired  targets. 

5.8.2. 5  Cued  Data  Quality  Control 

The  QC  implemented  throughout  the  cued  data  collection  included  the  following: 

•  IVS  measurements  before  and  after  each  day  of  production  measurements 

•  Function  tests  collected  whenever  background  measurements  were  collected 

•  Transmit  current  and  receiver  decay  monitoring 

•  Field  inversion  monitoring 

•  Re-collection  where  horizontal  target  location  was  offset  by  more  than  40  cm 

Throughout  the  course  of  the  cued  data  collection,  the  TEMTADS  system  was  tested  at  the  IVS 
on  a  twice  daily  basis  to  verify  sensor  functionality.  The  daily  IVS  measurements  were  inverted, 
and  the  extrinsic  parameters  (source  location)  and  intrinsic  parameters  (source  polarizabilities  [|3s]) 
results  were  monitored  and  recorded. 
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Figure  5-20.  TOAR  Dynamic  Data  with  Cued  Anomaly  Selections  in  Grids  78/46  and  79/46  (Western) 
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Figure  5-21.  TOAR  Dynamic  Data  with  Cued  Anomaly  Selections  in  Grids  82/47  and  83/47  (Eastern  Grids) 
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These  results  for  the  source  locations  are  presented  on  Figures  5-22.  All  but  one  of  the  source 
position  results  obtained  were  within  the  measurement  quality  objective  (MQO)  of  25  cm  for 
accuracy  (position  relative  to  the  ground  truth)  and  all  of  the  results  were  within  the  20  cm  MQO 
for  precision  (offset  from  the  average  position).  Because  the  accuracy  results  for  ISO  1  exhibit  a 
bias,  and  this  bias  is  not  present  in  the  remaining  ISOs,  it  is  assumed  that  the  MQO  failure  is 
attributable  in  part  to  inaccuracy  in  the  ground  truth  for  ISO-1. 


Figure  5-22.  TOAR  TEMTADS  Cued  Data  IVS  Dipole  Fit  Position  Results:  (Left)  Offset 
from  Ground  Truth  and  (Right)  Offset  from  Average  Position 

The  derived  source  (Is  were  assessed  by  performing  a  library  match  to  derive  decision  metrics 
(described  below  in  section  5.3.1)  for  each  measurement  and  the  results  are  presented  on 
Figure  5-23.  All  measurements  resulted  in  very  good  fits  to  their  respective  library  entries, 
indicating  proper  operation  of  the  system. 
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In  addition  to  the  twice-daily  IVS,  in-field  sensor  functionality  tests  were  performed  throughout 
each  survey  day  to  confirm  that  the  TEMTADS  system  components  were  functioning  within 
project  specifications.  Sensor  function  tests  were  performed  during  each  background  data 
collection  event.  The  sensor  function  test  results  are  shown  on  Figure  5-24,  Figure  5-25,  and 
Figure  5-26. 
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Variation 
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Figure  5-26.  TOAR  TEMTADS  Cued  Data  Sensor  Function  Test  Results  Tx  Current 
5.8.2.6  Cued  Data  Processing  and  Post-acquistion  Analysis 

The  data  processing,  analysis,  and  classification  steps  undertaken  to  generate  a  dig/no-dig  decision 
for  each  target  are  described  below.  These  steps  were  performed  for  the  TEMTADS  data  using  the 
UXA  module  within  the  Geosoft  processing  platform.  The  general  processing  flow  is  described  in 
the  following  steps: 

1.  Target  Association:  cued  measurements  are  associated  with  a  particular  target  from  the 
cued  target  list 

2.  Feature  Selection:  statistical  analyses  are  performed  to  determine  target  features 

3.  Library  Matching:  target  polarization  curves  extracted  for  each  target  are  matched  to  a 
library  of  munitions  known  to  exist  in  the  area  of  interest 

4.  Analyst  Calibration  Digs:  used  to  determine  the  source  of  any  identified  clusters  of  ‘like 
signatures’  as  well  as  to  inform  the  analyst  with  respect  to  the  placement  of  decision 
thresholds  in  the  prioritized  list 

5.  Prioritized  Target  List  Assembly:  final  AGC  determinations  on  the  set  of  cued  targets  and 
assembly  of  a  final  prioritized  dig  list 

Cued  data  were  imported  into  the  Geosoft  UXA  module  for  data  QC  and  inversion  modeling.  The 
data  were  levelled  using  background  data  collected  at  frequent  time  intervals  over  nearby, 
anomaly-free  background  locations.  The  measurements  used  for  background  correction  were 
reviewed  for  variability  and  to  identify  any  outliers  which  may  correspond  to  measurements  over 
subsurface  metal.  To  minimize  errors  in  the  background  removal  process,  spatial  and  temporal 
distance  between  the  background  and  target  measurements  were  minimized. 
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Target  data  were  inverted  using  both  single-source  and  multi-source  dipole  response  models  to 
estimate  target  parameters.  The  principle  parameters  of  interest  for  use  in  classification  of  the 
targets  were  the  three  polarizabilities  (pi,  P2,  and  P3)  estimated  for  each  target  by  UXA.  In 
addition  to  estimates  for  the  three  Ps  for  each  target,  an  estimated  location  and  depth  and  fit 
coherence  (i.e.,  the  correlation  between  the  observed  responses  and  the  model  predictions)  was 
also  returned  by  UXA  for  each  target  during  inversion. 

Classification  of  each  target  was  performed  using  the  intrinsic  features  (Ps)  derived  from  the 
single-source  and  multi-source  inversion  processes.  Classification  was  based  primarily  upon  how 
well  the  derived  Ps  matched  the  library  of  candidate  TOI  types.  The  final  composition  of  the  library 
was  informed  by  a  set  of  ACDs  (described  in  detail  in  Section  5. 6. 2. 8). 

5. 8. 2. 7  Library  Match  Decision  Metric 

Classification  is  based  primarily  on  the  goodness-of-fit  metric  (values  from  0.0  to  1.0)  generated 
by  UXA  during  a  comparison  of  the  P  values  estimated  for  each  surveyed  target  and  the  P  values 
in  the  site-specific  library  of  candidate  munitions.  This  comparison  is  performed  via  the  library 
match  utility  in  UXA.  The  goodness-of-fit  metric  is  a  measure  of  the  fit  correlation  between  a 
target  and  the  library  entry  that  best  fits  that  target,  with  higher  values  indicating  a  better  fit 
between  the  target  and  the  corresponding  item  in  the  library.  The  library  fit  analysis  matches  the 
following  four  combinations  of  Ps  to  those  of  the  candidate  library  TOIs: 

•  pi,  pl/p2,  pl/p3 

•  P1,P1/P2 

•  p l/p2,  pl/p3 

•  PI 

The  confidence  metrics  for  each  fit  combination  are  averaged  to  derive  a  ‘decision  metric’.  This 
library  matching  process  is  performed  for  each  single-solver  model  and  every  target  in  each  of  the 
multi-source  solver  candidate  realization  models.  For  each  flag  position,  the  best  library  fit  from  the 
single-solver  and  multi-solver  targets  is  used  as  the  decision  metric,  which  is  subsequently  used  to 
rank  and  classify  the  target  list.  Values  below  the  analyst  threshold  (nominally  0.825)  are  considered 
non-TOI.  The  analyst  threshold  was  refined  using  results  from  a  set  of  ACDs. 

5. 8. 2. 8  Analyst  Calibration  Digs 

Because  all  identified  anomalies  were  intrusively  investigated,  ACDs  were  not  performed 
separately  from  the  intrusive  investigation  phase.  Instead,  the  ACDs  were  simulated  through 
requests  for  ground  truth  from  the  ESTCP  Program  Office.  The  following  sections  describe  the 
rationale  for  the  ACD  selections. 

CLUSTER  ANALYSIS 

A  ‘cluster  analysis’  designed  to  identify  signatures  that  are  ubiquitous  to  the  site  was  performed 
using  the  UXA  signature  matching/cluster  identification  routines.  For  identified  clusters  that  were 
not  already  represented  in  the  library,  representative  samples  were  selected  for  addition  to  the  set 
of  ACDs  (in  this  case,  ‘selections’  provided  on  request  by  the  ESTCP  Program  Office  to  simulate 
the  ACD  process)  to  confirm  that  they  are  not  because  of  an  unexpected  TOI. 
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FEATURE  SPACE  ANALYSIS 

In  addition  to  the  library  match  decision  metric  described  above,  a  feature  space  analysis  was  also 
performed  to  identify  any  targets  that  did  not  match  a  specific  library  entry  but  had  the  combined 
characteristics  of  being  large,  rotationally  symmetric,  and  thick-walled.  Targets  identified  with 
these  characteristics  were  selected  and  added  to  the  ACD  selection  list. 

DIG/NO-DIG  THRESHOLD  CALIBRATION 

The  ACDs  were  also  used  to  finalize  the  analyst  threshold  (i.e.,  the  decision  metric  cutoff  value 
separating  the  prioritized  list  into  dig/no-dig  classifications).  These  targets  were  selected  by 
sampling  each  library  match  munition  type  above  and  below  the  initial  starting  metric  of  0.825. 
Because  the  analyst  threshold  must  be  set  low  enough  to  identify  TOIs  that  have  noisy 
polarizabilities  as  TOIs,  where  possible  targets  that  looked  qualitatively  like  a  TOI  were 
preferentially  selected.  Although  final  classification  is  based  upon  objective  numeric  criteria, 
qualitative  selection  of  these  threshold  calibration  digs  is  required  to  preferentially  select  those 
targets  that  are  likely  to  be  TOIs,  thus  calibrate  the  threshold  appropriately. 

LIBRARY  ENTRY  VERIFICATION 

The  initial  library  contained  a  comprehensive  list  of  munitions  including  entries  that  were  not  in 
the  list  of  munitions  expected  to  be  onsite  but  were  conservatively  left  in  as  a  representative 
size/shape  sample.  Where  one  or  more  of  these  entries  resulted  in  a  significant  number  of  matches, 
they  were  sampled  to  determine  if  they  were  actual  TOIs.  Where  these  samples  did  not  result  in 
the  recovery  of  a  TOI,  these  entries  were  removed  from  the  final  site-specific  list.  Selection  of 
targets  with  noise-free  polarizabilities  does  not  inform  the  threshold  calibration  because  if  they  are 
near  the  threshold  they  will  not  be  TOIs. 

5.8. 2.9  Candidate  TOI  Library 

The  initial  candidate  library  consisted  of  a  comprehensive  range  of  munitions  that  are  delivered 
with  the  UXA  installation  files  (listed  on  the  left  side  of  Figure  5-27).  The  final  site-specific 
library  of  candidate  TOIs  comprised  munitions  expected  to  be  onsite,  as  confirmed  by  the  results 
of  the  ACDs.  The  site-specific  library  had  one  or  more  entries  for  each  item  listed  on  the  right- 
hand  side  of  Figure  5-27. 
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Comprehensive  Library  (all  UXO) 


Type  Size 

Decay  [Symmetry 

Small  ISO 

0.21 

0.031 

1.02 

rifle  grenade 

0.79 

0.009 

1.03 

medium  ISO 

0.98 

0.046 

1.06 

large  ISO 

1.54 

0.054 

1.02 

81mm 

1.36 

0.032 

1.05 

75mm 

1.08 

0.057 

1.01 

60mm 

0.79 

0.038 

1.01 

5in  proj 

1.70 

0.058 

1.05 

57mm 

0.68 

0.056 

1.02 

40mm  grenade 

0.28 

0.092 

1.01 

Bin  stokes  mortar 

1.38 

0.070 

1.15 

3in  proj 

1.14 

0.052 

1.02 

37mm 

-0.08 

0.092 

1 

2.36in  rocket 

1.17 

0.075 

1.19 

155mm  proj 

2.10 

0.050 

1.13 

105mm  proj 

1.59 

0.059 

1.14 

20mm 

-0.30 

0.022 

0.98 

MK2  Hand  grenade 

0.32 

0.001 

1.04 

Fuze  Piece/part/component 

-0.45 

0.009 

1.1 

TOAR  Library  Entries 

Munition 

Size 

Decay 

Symmetry 

Small  ISO 

0.21 

0.030 

1.02 

Medium  ISO 

0.98 

0.050 

1.06 

Large  ISO 

1.54 

0.050 

1.02 

37-mm  projectile 

0.20 

0.060 

1.00 

75-mm  projectile 

1.08 

0.057 

1.01 

155-mm  projectile 

2.10 

0.050 

1.13 

Figure  5-27.  Initial  (left)  and  Final,  Site-specific  TOI  Library  (Right)  for  TOAR 
5.8.2.10  Intrusive  Activity,  Procedures,  and  Results 

CH2M  performed  intrusive  operations  from  13  September  through  1  October  2015.  All  prosecuted 
targets  were  investigated  and  documented  according  to  the  procedures  outlined  below. 


•  Reacquisition  of  targets.  Targets  were  previously  reacquired  by  the  PLS  for  the  cued 
interrogation  using  RTS.  The  pin  flag  was  located  approximately  50  cm  north  of  the  paint 
mark  denoting  the  target  location. 

•  Intrusively  investigate  the  anomaly.  Anomalies  were  excavated  to  30  cm  below  the 
expected  depth  below  ground  surface,  within  a  50  cm  radius  from  the  marked-out  anomaly 
location. 

•  Identify  recovered  item.  All  items  recovered  were  inspected  by  the  UXO  Safety  Officer 
and  Senior  UXO  Supervisor  to  ensure  that  each  item  was  properly  identified  and 
documented. 

•  MRSIMS  data  entry/whiteboard  and  photo.  Field  observations  of  each  recovered  item 
were  entered  into  CH2M’s  MRSIMS  field  tablets.  The  exact  location  and  depth  of  each 
item  was  recorded  by  a  CH2M  field  geophysicist  using  the  RTS.  Required  information  was 
written  onto  a  whiteboard,  and  a  photo  was  taken  with  the  item. 

•  Post-dig  clearance.  Before  declaring  a  dig  complete,  each  area  was  swept  with  a 
Schonstedt  magnetometer  and  a  White’s  All  Metal  detector  to  determine  if  any  additional 
items  remained. 

•  QC  check.  Approximately  10  percent  of  all  excavations  were  checked  by  UXO  QC 
personnel  with  the  Schonstedt  magnetometer  and  a  White’s  All  Metal  detector  to  ensure 
the  hole  was  clear. 
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•  Backfill  hole.  Once  the  excavation  was  declared  clear,  the  hole  was  backfilled  to  grade. 

All  of  the  TEMTADS  cued  targets  were  intrusively  investigated  as  well  as  a  set  of  MPV  targets 
located  in  the  100  percent  coverage  area.  Time  constraints  imposed  by  site  access  limitations  did 
not  allow  for  all  of  the  MPV  targets  outside  the  100  percent  coverage  area  to  be  investigated. 
During  the  intrusive  investigation,  all  of  the  QC  seeds  were  located  within  the  40  cm  MQO  and 
recovered.  The  results  of  the  intrusive  investigations  were  provided  to  the  ESTCP  Program  Office 
and  are  not  presented  in  this  report. 

5.8.2.11  Deliverables 

The  following  deliverables  resulted  from  the  data  collection  at  TOAR: 

1.  Dynamic  detection  data:  Raw  and  processed  dynamic  detection  data  were  provided  to  the 
ESTCP  Program  Office,  along  with  a  final  target  list  based  on  the  established  detection 
threshold 

2.  Cued  data:  Raw  sensor  data  (*.tem)  and  associated  GPS/IMEf  data  (*.gps) 

3.  Cross-reference  list:  A  text  readable  table  that  associates  TEMTADS  filenames  with  each 
Target  ID,  and  provides  any  applicable  collection  notes 

4.  Intrusive  results:  The  intrusive  results  were  provided  to  the  ESTCP  Program  Office  in  the 
form  of  a  table  listing  the  results  for  each  location  identified  for  intrusive  investigation  as 
well  as  photos  detailing  the  metallic  items  that  were  removed  from  each  location.  Note  that 
all  of  the  intrusive  results  were  firewalled  from  the  data  analyst  until  after  the  analysis  was 
completed  (with  the  exception  of  the  ACD  selections  described  above). 
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6.0  PERFORMANCE  ASSESSMENT  NBAFS 


The  performance  objectives  for  this  classification  survey  and  the  corresponding  results  are 
summarized  in  Table  6-1.  Details  on  the  results  for  each  objective  are  subsequently  discussed  in 
the  following  sections. 

Table  6-1.  NBAFS  Performance  Objectives  and  Results 


Performance 

Objective 

Metric 

Data 

Required 

Minimum 

Acceptable 

Criteria 

Nominal 

Success 

Criteria 

Result 

Data  Collection  Performance  Objectives 

Dynamic  survey 
spatial  coverage 

Effective 

footprint 

coverage 

Mapped 
survey  data 

95%  coverage  at  line 
spacing  of  60  cm  with 
no  gaps  greater  than 
20  cm  (80  cm 
between  adjacent  line 
centers) 

100% 

coverage  at 

60  cm  line 
spacing 

Pass  -  100%  of 
area  surveyed  at 
<  60  cm  lane 
spacing 

Along  line 
measurement 
spacing 

Point  to  point 
sample 
distance 

Mapped 
survey  data 

95%  <  20  cm 

98%  <  15  cm 

Pass  -  >95%  of 
mapped  survey  data 
<  20  cm  point  to  point 
sample  distance 

Detection  of  all 
TOI 

Percent  of 
seed  items 
detected 

Seed  item 
locations 

Geo-referenced 
anomaly  list 

100%  of  seeded 
items  within  a 
50-cm  halo 

100%  of 
seeded  items 
within  a  25 -cm 
halo 

Pass  -  100%  of 
seeded  items 
detected  within  a 
50-cm  halo 

Data  positioning 
repeatability 

Precision  of 
dipole -fit 
derived 
extrinsic  target 
features 

Target  fit 
positions  from 
daily  instrument 
verification  strip 
(IVS) 

Dynamic  positions 
+  20  cm 

Dynamic 
positions 
+10  cm 

Pass 

Sensor  response 
repeatability 
(dynamic  and 
cued  surveys) 

Precision  of 
dipole  fit 
derived 
intrinsic 
target 
features 

Dipole -fit 
derived  Ps 
from  daily 
IVS  data 

<  20%  RMS 
variation  in  P 
amplitudes 

<  10%  RMS 
variation  in  p 
amplitudes 

Dynamic:  Pass  - 
Amplitude  RMS 
variation  was  <20% 

Cued:  Pass  ~P 
Amplitude  RMS 
variation  was  <20% 

Cued 

interrogation 

anomaly 

coverage 

Instrument 

position 

Cued  data 

95%  of  anomalies 
where  the  center  of 
the  instrument  is 
positioned  within 

40  cm  of  actual 
target  location 

100%  of 
anomalies 
where  the 
center  of  the 
instrument  is 
positioned 
within  40  cm 
of  actual  target 
location 

Pass 
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Table  6-1.  NBAFS  Performance  Objectives  and  Results  (Continued) 


Performance 

Objective 

Metric 

Data 

Required 

Minimum 

Acceptable 

Criteria 

Nominal 

Success 

Criteria 

Result 

Data  Analysis/Classification  Performance  Objectives 

Maximize 

correct 

classification  of 
TOI 

Number  of 
TOI  correctly 
identified 

Ranked 
anomaly  lists 

Scoring  reports 
from  Institute 
for  Defense 
Analysis 

98%  of  all  seeded 
targets 

95%  of  all  TOI 

100%  of  all 
seeded  targets 

100%  of  all 
TOI 

Fail  -  see 
discussion  in 
Section  5.2 

Maximize 

correct 

classification  of 
non-TOI 

Number  of 
false  alarms 
eliminated 

Ranked 
anomaly  lists 

Scoring 
reports  from 
Institute  for 
Defense 
Analysis 

Reduction  of  clutter 
digs  required  by 
>50%  while 
retaining  all  TOI 

Reduction  of 
clutter  digs 
required  by 
>75%  while 
retaining  all  TOI 

In  retrospect  this 
was  not  achievable: 
70%  of  all  digs 
resulted  in  TOI. 

Model  results 
support 
classification 
decision 

Number  of 
anomalies 
classified  as 
‘Cannot 
Analyze’ 

Modeling  fit 
coherence 
results 

90%  of  targets  have 
fit  coherence  >  0.80 

95%  of  targets 
have  fit 
coherence 

>0.80 

N/A- 

classification  was 
not  performed 

Correct 
estimation  of 
target  parameters 

Accuracy  of 
estimated 
target 

locations  for 
seed  items 

Modeled  target 
parameters 

Results  of 
intrusive 
investigation 

X.Y  <  15  cm  (la) 

Z  <  10  cm  (la) 

X.Y  <  15  cm 
(la) 

Z  <  10  cm  (la) 

Fail  -  see 
discussion 

a  =  sigma  (standard  deviation) 

6.1  DYNAMIC  SURVEY  SPATIAL  COVERAGE 


The  TEMTADS  dynamic  detection  survey  was  designed  to  provide  100  percent  coverage  of  the 
investigation  area.  A  planned  transect  spacing  of  60  cm  was  used  to  ensure  sufficient  overlap  of 
the  80  cm  sensor  swath  footprint  between  traverses.  Before  the  dynamic  detection  survey  in  each 
grid,  1  m  wide  survey  lanes  were  established  using  ropes  held  in  place  by  non-metallic  tent  stakes. 
This  ensured  consistent  spacing  between  transects  and  even  coverage  of  the  investigation  area. 

The  success  criteria  for  this  objective  was  95  percent  coverage  of  the  investigation  area  with  no 
gaps  greater  than  80  cm  between  adjacent  lines.  This  objective  was  achieved,  as  no  gaps  greater 
than  80  cm  were  observed  in  the  data,  and  complete  coverage  of  the  investigation  area  was 
achieved. 
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6.2  ALONG-LINE  MEASUREMENT  SPACING 


The  TEMTADS  dynamic  detection  survey  was  designed  to  have  an  along-line  measurement 
spacing  of  15  cm  or  less  so  as  to  provide  a  dense  detection  dataset.  The  success  criteria  for  this 
objective  was  for  98  percent  of  mapped  data  points  to  be  within  15  cm  of  the  along-line 
neighboring  data  point,  and  for  no  more  than  5  percent  of  the  data  points  to  be  outside  of  20  cm 
from  the  along-line  neighboring  data  point.  This  objective  was  achieved,  as  no  along-line  spacing 
greater  than  15  cm  was  observed  in  the  data. 

6.3  DETECTION  OF  ALL  TOI 

This  objective  involved  the  detection  of  TOI  during  the  dynamic  detection  survey.  This  objective 
was  verified  using  a  blind  seeding  program.  Before  the  dynamic  detection  survey,  110  seed  items 
were  buried  within  the  investigation  area.  These  seed  items  consisted  of  Small  ISO  Schedule  80s, 
0.5-inch  x  2-inch  20-mm  simulants,  0.5-inch  x  3-inch  20-mm  simulants,  and  inert  20-mm  projectiles. 
The  20-mm  and  20-mm  simulants  were  buried  at  depths  up  to  15  cm,  and  Small  ISO  (Schedule  80) 
were  buried  at  depths  up  to  30  cm,  as  documented  in  the  Demonstration  Seed  Plan. 

6.4  DATA  POSITIONING  REPEATABILITY 

This  objective  involved  twice  daily  surveys  of  the  seeded  items  in  the  IVS  to  show  that  the  data 
positioning  was  repeatable  to  better  than  0.2  m.  The  positions  of  the  seeded  items  in  the  dynamic 
survey  were  determined  as  the  locations  of  the  amplitude  response  anomaly  peaks  associated  with 
the  seeded  items.  All  detections  were  within  the  specified  0.2  m  tolerance.  The  results  are 
summarized  in  Table  6-2. 

Table  6-2.  NBAFS  Dynamic  IYS  Positioning  Results 


Seed  Item 

Maximum  Error  (m) 

Average  Error  (m) 

M55A3B1  20-mm 

0.15 

0.06 

2.25-inch  Rocket 

0.17 

0.04 

Small  ISO80 

0.17 

0.07 

The  positions  of  the  seeded  items  surveyed  during  the  cued  investigation  were  derived  from  the 
dipole  fit  analysis  of  the  measured  data.  These  results  are  summarized  in  Table  6-3.  One  of  the 
results  exceeded  the  stated  tolerance,  but  the  cause  for  the  exceedance  was  a  base  station  battery 
failure  that  occurred  just  before  the  very  last  measurement  of  the  day  (and  after  the  other  two  IVS 
items  had  been  successfully  surveyed  at  the  end  of  the  day).  Thus  the  failure  was  deemed  to  have 
no  adverse  implications  for  the  usability  of  the  day’s  data. 

Table  6-3.  NBAFS  Static  IYS  Positioning  Results 


Seed  Item 

Maximum  Error  (m) 

Average  Error  (m) 

M55A3B1  20-mm 

0.10 

0.05 

2.25-inch  Rocket 

0.13 

0.08 

Small  ISO80 

0.36 

0.10 
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6.5  SENSOR  RESPONSE  REPEATABILITY 


This  objective  involves  the  repeatability  of  sensor  response  amplitude  over  the  course  of  the 
project  for  each  seed  item  buried  at  the  IVS.  Consistent  sensor  responses  can  only  be  obtained  if 
the  sensor  is  functioning  properly.  Sensor  response  amplitudes  for  each  IVS  data  collection  event 
were  tracked  throughout  the  life  of  the  project  for  both  the  dynamic  and  cued  phases  of  data 
collection.  The  objective  was  considered  to  be  met  if  there  was  less  than  20  percent  RMS  deviation 
in  response  amplitudes  for  dynamic  and  cued  IVS  results. 

Results  for  the  dynamic  data  responses  are  detailed  in  Table  6-4.  The  largest  RMS  deviation  was 
for  the  20-mm,  at  9.9  percent,  which  means  that  this  objective  was  achieved  for  the  dynamic  data 
phase  of  the  demonstration. 

Table  6-4.  NBAFS  Dynamic  Data  Sensor  Response  Results 


Seed  Item 

Average  Response  (mV/A) 

%  RMS  Deviation 

:  M55A3B1  20-mm 

9.6 

9.9 

2.25-inch  Rocket 

147.2 

4.6 

Small  ISO80 

28.4 

6.5 

Results  for  the  cued  P  amplitudes  are  detailed  in  Table  6-5.  The  P  amplitudes  for  the  IVS  seed 
items  were  within  the  20  percent  RMS  tolerance  established  for  this  performance  objective. 


Table  6-5.  NBAFS  Cued  Data  Sensor  Response  Results 


Seed  Item 

Average  p  Amplitude 

%  RMS  Deviation 

M55A3B1  20-mm 

0.2 

4.8 

2.25-inch  Rocket 

44.3 

4.6 

Small  ISO80 

1.7 

3.3 

6.6  CUED  INTERROGATION  OF  ANOMALIES 

This  objective  requires  that  95  percent  of  anomalies  result  in  having  the  target  fit  location  within 
a  0.4  m  horizontal  radius  from  the  center  of  the  array.  This  is  required  to  ensure  that  the  target  is 
properly  energized  along  its  three  principal  axes.  This  metric  was  not  achieved  at  NBAFS  -  only 
75  percent  of  anomalies  resulted  in  target  fit  locations  within  the  stated  radius.  This  was  because 
of  the  fact  that  the  majority  of  the  measurements  were  made  at  locations  with  multiple  targets  in 
view.  The  field  team  used  a  single  target  inversion  routine  to  reposition  the  sensor  if  the  target  was 
not  within  the  specified  radius.  In  a  multi-target  scenario,  the  real-time  feedback  is  imprecise  and 
there  is  no  way  to  ensure  that  the  most  likely  TOI  will  be  within  the  0.4  m  radius  of  the  sensor. 
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6.7  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 


Not  applicable  (see  classification  discussion  above). 

6.8  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

Not  applicable  (see  classification  discussion  above). 

6.9  MODEL  RESULTS  SUPPORT  CLASSIFICATION 

This  objective  requires  that  the  model  derived  during  the  inversion  process  must  match  the 
observed  data  to  within  a  correlation  of  0.8.  This  metric,  provided  by  UXA  as  the  ‘fit  coherence’ 
(UXA_fit_coh),  is  an  output  of  the  dipole  analysis  inversion  routine.  98  percent  of  the  inversion 
results  resulted  in  fit  coherence  greater  than  0.8. 

6.10  MODEL  RESULTS  SUPPORT  CLASSIFICATION 

This  objective  involved  assessing  the  accuracy  of  the  derived  target  parameters  by  comparing  the 
seed  ground  truth  with  the  positions  derived  from  the  inversion  process.  There  were  very  few  seeds 
that  were  investigated  (because  of  the  fact  that  the  seeds  were  originally  distributed  over  the  entire 
site  and  only  a  small  section  of  the  site  was  subject  to  cued  investigation).  Given  the  failure  of  the 
derived  signatures  to  accurately  predict  the  anomaly  sources  when  more  than  three  sources  were 
recovered,  and  the  fact  that  33  percent  of  the  flags  had  four  or  more  sources  recovered,  it  was 
determined  that  any  analysis  of  performance  against  the  few  emplaced  seeds  that  were  measured 
would  add  no  value  to  the  demonstration. 
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7.0  PERFORMANCE  ASSESSMENT  TOAR 


The  performance  objectives  for  this  classification  survey  and  the  corresponding  results  are 
summarized  in  Table  7-1.  Details  on  the  results  for  each  objective  are  subsequently  discussed  in 
the  following  sections. 

Table  7-1.  TOAR  Performance  Objectives  and  Results  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Minimum  Acceptable 
Criterion 

Result 

Dynamic 
survey  spatial 
coverage 

Effective  footprint 
coverage 

Mapped  survey  data 

100%  at  <  75  cm  cross-track 
measurement  spacing  with 
intended  spacing  of  50  cm 

Fail  (pass  for  100% 
coverage  area) 

Along-line 

measurement 

spacing 

Point  to  point 
sample  distance 

Mapped  survey  data 

98%  <  25  cm;  no  gaps 
>  40  cm  unless  obstruction  or 
hazard  is  present 

Fail  (pass  for  100% 
coverage  area) 

Detection  of 
all  TOIs 

Percent  of  seed 
items  detected 

Seed  item  locations 
Georeferenced  anomaly 
list 

100%  of  seed  items  within  a 
40  cm  radius  of  ground  truth 

Fail  (all  seed  items  were 
detected  but  two  were  not 
within  the  specified  distance 
because  of  gaps  in  coverage) 

Initial  dynamic 
survey  data 
l  positioning 

Accuracy  of 
derived  target 
positions 

Derived  target  positions 
from  initial  measurements 
at  the  IVS 

Derived  positions  within 

25  cm  of  the  ground  truth 

Fail  (one  outlier  from 
horizontal  targets-at  3 1  cm) 

Ongoing 
dynamic 
survey  data 
positioning 

Precision  of 
derived  target 
positions 

Derived  target  positions 
from  daily  measurements 
at  the  IVS 

Derived  positions  within 

25  cm  of  the  average 
positions  during  ongoing 
daily  measurements 

Fail  (two  outliers  from 
horizontal  targets — all 
within  30  cm) 

Initial  cued 
survey  data 
positioning 

Accuracy  of 
dipole-fit-derived 
target  positions 

Target  fit  positions  from 
initial  measurements  at 
the  IVS 

IV S  item  fit  locations  within 
25  cm  of  ground  truth 
locations 

Pass 

Ongoing  cued 
survey  data 
positioning 

Precision  of 
dipole-fit-derived 
target  positions 

Target  fit  positions 
from  daily 
measurements  at  the 
IVS 

IVS  item  fit  locations  within 
+20  cm  of  average  fit 
locations  during  ongoing 
daily  measurements 

Pass 

Initial  cued 

sensor 

polarizability 

accuracy 

Accuracy  of 
dipole-fit-derived 
intrinsic  target 
features 

Dipole-fit-derived 
polarizabilities  from 
initial  measurements  at 
the  IVS 

Library  match  metric  >0.9  to 
initial  polarizabilities  for 
each  set  of  inverted 
polarizabilities 

Pass 

Ongoing  cued 
sensor 

polarizability 

precision 

Precision  of 
dipole-fit-  derived 
intrinsic  target 
features 

Dipole -fit-derived 
polarizabilities  from 
daily  measurements  at 
the  IVS 

Match  metric  >0.95  to  initial 
polarizabilities  at  the  IVS  for 
each  set  of  inverted 
polarizabilities  from  daily 
measurements 

Pass 

Cued 

interrogation 

anomaly 

coverage 

Instrument 

position 

Cued  data 

100%  of  anomalies  where 
the  center  of  the  array  is 
positioned  within  30  cm  of 
anomaly  location 

Pass 

Correct 
classification 
of  TOIs 

Number  of  TOIs 
correctly 
identified 

Ranked  anomaly  lists 
Scoring  reports  from 
ESTCP  Program  Office 

100%  of  all  seed  targets 
100%  of  all  TOIs  categorized 
as  ‘dig’  or  ‘cannot  analyze’ 

Pass 

Model  results 
support 
classification 
j  decision 

Number  of 
anomalies 
classified  as 
‘cannot  analyze’ 

Modeling  fit  coherence 
results 

>90%  of  targets  have  fit 
coherence  >  0.80 

Pass 
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7.1  DYNAMIC  SURVEY  SPATIAL  COVERAGE 


The  TEMTADS  dynamic  detection  survey  was  designed  to  provide  100  percent  coverage  of  the 
investigation  area.  A  planned  transect  spacing  of  50  cm  was  used  to  ensure  sufficient  overlap  of 
the  80  cm  sensor  swath  footprint  between  traverses.  Data  were  collected  continuously  with  an  even 
walking  pace;  this  consistent  pace  allowed  the  RTS  base  to  reestablish  contact  with  the  RTS  prism 
on  the  TEMTADS  after  it  emerged  from  behind  an  obstacle.  Using  this  approach,  two  types  of 
gaps  exist  in  this  data  set:  one  caused  by  the  RTS  ‘gun’  shadow  created  when  the  TEMTADS  is 
carried  behind  a  tree,  and  one  created  by  obstacles  such  as  downed  trees,  upright  trees,  and 
boulders.  For  the  case  of  the  RTS  shadow,  data  were  not  interpolated  even  though  the  dynamic 
TEMTADS  data  were  still  being  collected  and  there  were  positioning  data  on  either  side  of  the 
tree,  the  positioning  data  were  not  interpolated  in  processing.  As  in  any  survey,  obstacles  such  as 
trees  and  boulders  precluded  coverage.  To  achieve  the  stated  goal  of  100  percent  coverage,  gaps 
because  of  interruption  of  the  RTS  data  required  re-collection,  whereas  gaps  because  of  physical 
obstructions  were  documented  as  such. 

As  a  result  of  time  constraints,  the  gap  fill  required  to  achieve  the  100  percent  coverage  goal  was 
performed  on  only  one  of  the  four  grids  (grid  82/47).  Coverage  for  this  grid  was  81.2  percent 
before  gap  fill  and  94. 1  percent  post  gap  fill  because  of  the  elimination  of  gaps  associated  with  the 
RTS.  The  5.9  percent  of  the  grid  not  mapped  was  the  result  of  physical  obstructions;  thus  the 
100  percent  coverage  metric  was  achieved  for  this  grid.  Grids  78/46  and  79/46  (western  grids) 
combined  had  an  overall  coverage  of  72.4  percent  with  no  gap  re-collection.  The  gap  percentage 
because  of  obstacles  was  not  calculated  because  the  locations  of  these  objects  were  not  recorded. 
Grids  82/47  and  83/47  (eastern  grids)  combined  had  71.2  percent  coverage,  not  accounting  for 
obstacles. 

7.2  ALONG-LINE  MEASUREMENT  SPACING 

The  TEMTADS  dynamic  detection  survey  was  designed  to  have  an  along-line  measurement 
spacing  of  15  cm  or  less  so  as  to  provide  a  dense  detection  dataset.  The  success  criterion  for  this 
objective  was  for  98  percent  of  mapped  data  points  to  be  within  25  cm  of  the  along-line 
neighboring  data  point.  This  objective  was  not  achieved  as  gaps  in  coverage  created  by  the  RTS 
‘gun’  shadow  were  not  interpolated.  The  heavily  wooded  site  inhibited  the  continuous  contact 
between  the  base  RTS  and  the  prism  located  on  the  TEMTADS. 

7.3  DETECTION  OF  ALL  TOI 

This  objective  involved  the  detection  of  TOIs  during  the  dynamic  detection  survey  using  the  blind 
seeding  program.  Before  the  dynamic  detection  survey  20  seed  items  were  buried  within  the 
investigation  area.  These  seed  items  consisted  of  small  schedule  80  ISOs  and  medium  schedule  40 
ISOs  buried  at  depths  up  to  17  cm.  The  minimum  acceptable  criterion  of  100  percent  of  seed  item 
locations  being  predicted  within  a  40  cm  radius  of  ground  truth  was  not  met  (Table  7-2).  With  the 
exception  of  Seed  1  and  Seed  4,  all  other  seed  items  were  detected  within  the  40  cm  radius  from 
the  recorded  PLS  position.  The  detection  distance  for  Seeds  1  and  4  was  outside  of  the  MQO 
because  of  the  gaps  in  coverage  discussed  in  Section  7.1.  All  of  the  seeds  in  the  100  percent 
detection  grid  passed  the  40  cm  MQO. 
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Table  7-2.  TOAR  TEMTADS  QC  Seed  Detection 


Seed  ID 

Delta  X  (m) 

Delta  Y  (m) 

Distance  (m) 

1 

-0.56 

0.00 

0.56 a 

2 

0.03 

0.21 

0.22 

3 

-0.13 

-0.20 

0.24 

4 

-0.17 

-0.39 

0.43 a 

5 

0.34 

-0.10 

0.35 

6 

0.08 

0.03 

0.09 

7 

-0.11 

0.05 

0.13 

8 

0.12 

-0.16 

0.20 

9 

0.13 

-0.06 

0.14 

10 

-0.31 

-0.06 

0.32 

11 

0.10 

-0.28 

0.30 

12 

0.16 

0.28 

0.32 

13 

0.18 

-0.01 

0.18 

14 

-0.05 

-0.06 

0.08 

15 

0.07 

-0.13 

0.14 

16 

-0.24 

-0.02 

0.24 

17 

0.13 

-0.11 

0.17 

18 

0.08 

0.16 

0.18 

19 

0.08 

-0.06 

0.10 

20 

0.29 

0.12 

0.32 

a  Outside  the  MQO. 


7.4  INITIAL  DYNAMIC  IYS  SURVEY  DATA  POSITIONING 

This  objective  was  to  demonstrate  positioning  accuracy  during  initial  dynamic  data  collection  over 
the  IVS  with  the  TEMTADS  by  deriving  positions  within  25  cm  of  ground  truth.  The  metric  was 
assessed  by  evaluating  the  derived  target  positions  from  the  initial  data  collection  at  the  IVS  as 
determined  by  the  amplitude  response  anomaly  peaks  associated  with  the  seeded  items.  Results 
for  the  dynamic  data  responses  are  provided  in  Table  7-3.  The  largest  deviation  was  39  cm  for 
ISO-1,  which  means  that  this  objective  was  not  achieved  for  all  ISOs. 

Table  7-3.  TOAR  TEMTADS  Initial  Dynamic  IVS  Survey  Positioning  Results 


Seed  Item 

Error (m) 

ISO-1 

0.39 

ISO-2 

0.25 

ISO-3 

0.25 

ISO-4 

0.08 
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7.5  ONGOING  DYNAMIC  SURVEY  DATA  POSITIONING 


This  objective  involved  the  repeatability  of  sensor  response  amplitude  over  the  course  of  the 
project  for  each  seed  item  buried  at  the  IVS.  The  minimum  success  criterion  was  that  the  derived 
positions  of  the  IVS  targets  would  be  within  25  cm  of  the  average  positions  derived  from  the 
ongoing  daily  IVS  measurements.  This  approach  gives  an  estimate  of  the  precision  of  the 
positioning  and  is  independent  of  any  ground  truth  errors.  Results  for  the  dynamic  data  responses 
are  detailed  in  Table  7-4.  The  largest  deviation  was  30  cm  for  ISO-2,  which  means  that  this 
objective  was  not  fully  achieved  for  the  dynamic  data  phase  of  the  demonstration.  Figure  7-1 
shows  the  dynamic  IVS  position  precision  graphically. 


Table  7-4.  TOAR  TEMTADS  Dynamic  IVS  Positioning  Results 


Seed  Item 

Maximum  Error  (m) 

Average  Error  (m) 

ISO-1 

0.23 

0.15 

ISO-2 

0.30 

0.20 

ISO-3 

0.29 

0.18 

ISO-4 

0.18 

0.10 

0.30 
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Figure  7-1.  TOAR  Dynamic  IVS  Position  Precision  (Errors  Relative  to  Average  Derived 

Positions) 
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7.6  INITIAL  CUED  SURVEY  DATA  POSITIONING 


This  objective  was  to  demonstrate  positioning  accuracy  during  initial  cued  data  collection  with  the 
TEMTADS.  The  metric  was  assessed  by  evaluating  the  dipole-fit-derived  target  positions  from 
initial  measurements  at  the  IVS.  The  minimum  success  criterion  was  that  100  percent  of  the  fit- 
derived  positions  of  the  IVS  targets  would  be  within  25  cm  of  the  ground  truth  location  for  the 
initial  IVS  measurements.  This  objective  was  achieved  for  the  initial  IVS  cued  survey  (Table  7-5). 

Table  7-5.  TOAR  TEMTADS  Initial  Cued  IVS  Survey  Positioning  Results 


Seed  Item 

Error (m) 

ISO-1 

0.19 

ISO-2 

0.16 

ISO-3 

0.06 

ISO-4 

0.18 

7.7  ONGOING  CUED  SURVEY  DATA  POSITIONING 

This  objective  was  to  demonstrate  positioning  precision  during  ongoing  cued  data  collection  with 
the  TEMTADS.  The  metric  was  assessed  by  evaluating  the  dipole-fit-derived  target  positions  from 
daily  measurements  at  the  IVS.  The  minimum  success  criterion  was  that  the  derived  positions  of 
the  IVS  targets  be  within  20  cm  of  the  average  positions  derived  from  the  ongoing  daily  IVS 
measurements.  As  expected,  the  precision  of  the  measurements  was  better  than  the  initial  accuracy 
assessment  because  the  initial  accuracy  assessment  factors  in  ground  truth  measurement  error.  The 
performance  criterion  for  this  metric  was  met  (Table  7-6).  Results  for  the  cued  data  responses  are 
detailed  in  Figure  7-2. 


Table  7-6.  TOAR  TEMTADS  Cued  IVS  Positioning  Results 


Seed  Item 

Maximum  Error  (m) 

Average  Error  (m) 

ISO-1 

0.10 

0.04 

ISO-2 

0.12 

0.05 

ISO-3 

0.12 

0.05 

ISO-4 

0.12 

0.05 
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Figure  7-2.  TOAR  Cued  IVS  Position  Precision  (Errors  Relative  to  Average  Derived 

Positions) 

7.8  INITIAL  CUED  MEASUREMENT  POLARIZABILITIES 


This  objective  was  to  demonstrate  that  the  derived  polarizabilities  of  the  IVS  targets  matched  the 
polarizabilities  in  the  library  during  the  initial  IVS  measurements.  The  minimum  success  criterion 
was  met,  as  the  dipole-fit-derived  polarizabilities  for  the  IVS  targets  during  initial  IVS 
measurements  exhibited  a  decision  metric  of  >0.9  for  all  IVS  ISOs.  These  results  are  provided  as 
the  first  day’s  data  points  on  Figure  7-3. 
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Figure  7-3.  TOAR  Decision  Metric  for  IVS  ISOs 
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7.9  ONGOING  CUED  SENSOR  POLARIZABILITY  PRECISION 


This  objective  was  to  demonstrate  that  the  derived  polarizabilities  of  the  IVS  targets  from  ongoing 
daily  IVS  measurements  matched  the  polarizabilities  derived  from  the  initial  IVS  measurements. 
The  minimum  success  criterion  was  that  the  ongoing  daily  decision  metric  for  the  IVS  targets  was 
>0.95.  Although  the  initial  IVS  signature  for  each  ISO  can  be  used  for  this  test,  the  matches  to  the 
existing  library  entry  were  sufficient  to  pass  the  MQO,  so  it  was  not  necessary  to  use  the  initial 
IVS-derived  signatures. 

7.10  CUED  INTERROGATION  ANOMALIES 

To  collect  data  that  support  classification,  the  source  of  the  anomaly  must  be  illuminated  along  its 
three  principle  axes.  To  ensure  this,  the  sensor  array  must  be  positioned  directly  over  the  center  of 
the  item.  The  metric  for  this  objective  was  to  demonstrate  that  the  center  of  the  array  was  within 
sufficient  distance  of  the  anomaly  source’s  location  during  cued  interrogation.  Positions  of  the 
array  center  were  derived  from  the  RTS  and  IMU  data  along  with  the  derived  target  locations  for 
comparison  against  each  other.  The  array  center  positions  were  compared  against  the  supplied 
target  coordinates  as  part  of  the  daily  QC  process  during  data  collection.  The  minimum  and 
nominal  success  criterion  for  this  objective  was  that  100  percent  of  the  final  derived  targets  be 
positioned  within  40  cm  of  the  center  of  the  array.  Exceptions  include  targets  that  are  considered 
‘cannot  analyze’  (e.g.,  saturated  response  area)  and  multi-target  sources.  Thirty-six  targets  were 
classified  as  ‘cannot  analyze’  because  the  source  was  too  far  from  the  center  of  the  array  to  support 
classification.  All  of  the  targets  not  considered  ‘cannot  analyze’  had  a  valid  measurement  within 
this  metric. 

7.11  CORRECT  CLASSIFICATION  OF  TARGETS  OF  INTEREST 

Meeting  this  objective  was  a  primary  key  measure  of  the  effectiveness  of  the  classification 
approach.  By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter 
estimation  and  classification  algorithms,  targets  would  be  classified  with  high  efficiency.  The 
metric  for  this  objective  demonstrates  that  TOIs  are  correctly  classified  as  TOIs  on  the  final  ranked 
anomaly  list.  The  ranked  anomaly  list  was  submitted  to  ESTCP  for  scoring  against  the  emplaced 
seed  items  and  the  intrusive  results.  The  minimum  success  criterion  was  correct  classification  of 
100  percent  of  the  seed  items  and  native  TOIs  as  TOIs.  Successful  achievement  of  this  metric 
would  include  seed  items  and  other  TOIs  categorized  as  ‘dig’  or  ‘cannot  analyze’  on  the  final 
ranked  anomaly  list.  A  pseudo  receiver  operating  characteristic  (ROC)  curve  presenting  the 
classification  performance  is  shown  in  Figure  7-4.  The  ROC  curve  is  derived  by  moving  down 
the  prioritized  list  and  adding  1  to  the  y  axis  for  each  recovered  TOI  and  1  to  the  x  axis  for  each 
non-TOI  recovery.  A  ROC  curve  representing  perfect  classification  rises  vertically  until  all  TOIs 
are  identified,  then  horizontally  for  the  remaining  non-TOI  results.  A  diagonal  ROC  curve 
indicates  no  classification  performance.  The  results  presented  in  Figure  7-4  indicate  good 
classification  performance.  All  TOIs  were  correctly  categorized  as  ‘dig’;  six  TOIs  were  initially 
selected  as  Category  -1  (ACDs),  and  the  remaining  26  TOIs  were  all  selected  as  Category  1  (high 
likelihood  TOI). 
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7.12  MODEL  RESULTS  SUPPORT  CLASSIFICATION  DECISION 

This  objective  was  to  demonstrate  that  data  gathered  exhibit  a  measure  of  the  correlation  between 
the  model  and  the  observed  data,  which  is  used  to  determine  whether  the  model  will  support 
classification.  The  metric  used  to  validate  that  the  model  responses  support  classification  is 
the  percentage  of  targets  that  cannot  be  classified  as  well  as  the  fit  coherence  between  the  model 
response  and  observed  data.  The  targets  that  cannot  be  classified  were  identified  as  having  a  ‘fit 
coherence’  of  less  than  0.8.  (Fit  coherence  is  an  output  of  the  UXA  dipole  fit  routine  indicating 
the  correlation  of  the  modelled  data  with  the  observed  responses).  The  minimum  success  criterion 
for  the  number  of  targets  with  responses  that  support  classification  was  that  >90  percent  of  the 
targets  meet  the  fit  coherence  requirement.  Two  targets  were  categorized  as  ‘can’t  analyze’ 
because  of  fit  coherence,  resulting  in  99.6  percent  of  the  targets  meeting  this  metric. 
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8.0  COST  ASSESSMENT 


ESTCP  projects  are  required  to  develop  and  validate,  to  the  extent  possible,  the  expected  operational 
costs  of  the  technology.  The  intent  of  this  section  is  to  identify  the  information  that  was  tracked  or  the 
data  that  were  obtained  during  the  demonstrations  that  will  aid  in  establishing  realistic  costs  for 
implementing  the  technology  and  comparing  it  to  potential  alternative  technologies. 

One  of  the  driving  factors  for  applying  classification  at  munitions  response  sites  is  more  efficient  use 
of  available  resources,  which  can  be  used  to  clean  up  more  land  more  quickly  when  intrusive 
investigation  of  non-munitions  targets  is  reduced.  The  actual  costs  of  these  demonstrations  included 
extensive  planning,  reporting,  and  coordination,  as  well  as  redundant  data  collection  and  processing 
as  these  processes  had  not  been  standardized  at  the  time  of  the  demonstrations.  The  costs  presented 
below  may  not  be  representative  of  what  would  be  expected  for  production  application. 

8.1  COST  MODEL 

The  tracked  costs  for  NBAFS  and  TOAR  are  provided  in  Table  8-1  and  Table  8-2,  respectively. 
The  costs  are  shown  graphically  as  Figure  8-1  for  NBAFS  and  Figure  8-2  for  TOAR.  Note  that 
the  TEMTADS  system  was  provided  by  NRF  at  no  cost  to  CH2M  for  these  demonstrations.  The 
costs  per  acre  (for  dynamic  data  collection)  and  per  anomaly  (for  cued  data  collection)  do  not 
include  the  costs  that  would  be  associated  with  procurement  of  the  sensor. 

Table  8-1.  NBAFS  Costs  for  TEMTADS 


Cost  Element 

Tracked  Data 

Cost/Quantity 

Site  Setup 

General  site  setup 

Costs  for  planning,  mobilization,  general  site  setup,  shipping 
of  equipment,  surface  sweep,  QC  seeding,  surveyor  services, 
onsite  archaeologists,  and  demobilization. 

$92,000 

Dynamic  TEMTADS  Survey  Costs 

Dynamic  detection  survey 
and  data  processing 

Dynamic  detection  survey  (6  acres),  including: 

Field  labor  (three  geophysicists/geophysical  technicians,  one 
UXO  technician),  equipment  setup,  equipment  rentals,  IVS 
setup  and  data  collection,  data  processing,  per  diem, 
archaeological  oversight 

$57,000 

($9500/acre) 

Cued  TEMTADS  Survey  Costs 

Reacquisition  of  cued 
targets 

Reacquisition  of  anomalies  for  cued  surveys,  cued  surveys 
(1500  anomalies),  including: 

Field  labor  (three  geophysicists/geophysical  technicians,  one 
UXO  technician),  equipment  setup,  equipment  rentals,  IVS 
setup  and  data  collection,  per  diem,  archaeological  oversight 

$110,000 
($73.3 1/anomaly) 

Cued  survey  data 
processing 

Processing  of  cued  data,  including  several  site  visits  for  task 
kickoff  and  quality  control  purposes 

$64,000 

( $42 .66/anomaly) 

Intrusive  Investigation 

Intrusive  Investigation 
Costs 

Reacquisition  of  anomalies  and  all  UXO  team  related  costs 
related  to  the  intrusive  investigation  and  documentation  of 
discoveries  per  the  ESTCP  intrusive  investigation  instructions 

$228,000 

($598. 16/anomaly) 
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NBAFS  Cost  Breakdown 
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Figure  8-1.  NBAFS  Cost  Breakdown  Graph 


Table  8-2.  TOAR  Costs  for  TEMTADS 


Cost  Element 

Tracked  Data 

Cost/Quantity 

Site  Setup 

General  site  setup 
activities 

Costs  for  planning,  mobilization,  general  site  setup,  shipping  of 
equipment,  instrument  aided  visual  surface  sweep,  QC  seeding, 
surveyor  services,  and  demobilization 

$63,000 

MPPEH  and  MEC 
activities 

MPPEH  management  and  MEC  disposal  for  the  instrument-aided 
visual  surface  clearance  was  de-scoped  from  CH2M  and  passed 
along  to  the  removal  action  MEC  contractor 

$5,300 

Dynamic  TEMTADS  Survey  Costs 

Dynamic  detection  survey 

Dynamic  detection  survey  (0.89  acre),  including  field  labor  (three 
geophysicists/  geophysical  technicians,  one  UXO  technician), 
equipment  setup,  equipment  rentals,  IVS  setup  and  data  collection, 
data  processing,  and  per  diem 

$77,500 
($87,078/acre) a 

Dynamic  detection  data 
processing 

Dynamic  detection  survey  data  processing 

$8,300 

Cued  TEMTADS  Survey  Costs 

Reacquisition  of  cued 
targets 

Reacquisition  of  anomalies  for  cued  surveys  (497  anomalies), 
including  field  labor  (three  geophysicists/  geophysical  technicians, 
one  UXO  technician),  equipment  setup,  equipment  rentals,  IVS 
setup,  data  collection,  and  per  diem 

$40,000 

($80.48/anomaly) 

Cued  survey  data 
processing 

Processing  of  cued  data,  including  several  site  visits  for  task  kickoff 
and  quality  control  purposes 

$6,400 

($12. 88/anomaly) 

Intrusive  Investigation 

Reacquisition  of 
anomalies  for  intrusive 
investigation 

Reacquisition  of  anomalies  and  all  UXO  team-related  costs  related 
to  the  intrusive  investigation  and  documentation  of  discoveries  per 
the  ESTCP  intrusive  investigation  instructions 

$173,200 

($348.49/anomal 

y) 

Management  of  MPPEH 
and  MEC  disposal 

Management  of  material  potentially  presenting  an  explosive  hazard 
and  MEC  disposal  for  the  intrusive  investigation  was  de-scoped 
from  CH2M  and  passed  along  to  removal  action  MEC  contractor 

$11,600 

a  See  implementation  issues  section  for  explanation  of  high  per-acre  cost. 
MPPEH  =  Material  Potentially  Presenting  an  Explosive  Hazard 
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TOAR  Cost  Breakdown 


$600,000 

H  $500,000 

$400,000 

o 

u 

£  $300,000 
a> 

O  $200,000 
E 

o  $100,000 
$0 


$385,300 


Site  setup  Dynamic  Cued  survey  &  Intrusive  Total  Cost 
survey  &  data  data  Investigation 

processing  processing 


Figure  8-2.  TOAR  Cost  Breakdown  Graph 


8.2  COST  DRIVERS  AND  BENEFITS 

For  a  typical  munitions  response  project,  over  75  percent  of  the  budget  is  spent  on  the  removal  of 
non-hazardous  scrap  items.  That  three  quarters  of  the  budget  is  spent  on  scrap  removal  is  driven  by 
the  fact  that  DMM/UXO  make  up  1  percent  or  less  of  the  metal  items  on  a  munitions  site.  Therefore, 
a  reduction  in  the  number  of  scrap  items  that  must  be  treated  as  potentially  hazardous  has  a  large 
impact  on  the  budget.  Classification  using  advanced  technology,  such  as  the  TEMTADS,  has  been 
proven  to  greatly  reduce  the  number  of  scrap  items  that  require  intrusive  investigation. 

Based  on  the  results  of  the  NBAFS  demonstration,  the  TEMTADS  was  found  to  be  unsuitable  for 
the  site  conditions  encountered  and  a  comparison  against  potential  alternatives  is  not  warranted 
(because  the  technology  demonstrated  is  not  a  viable  alternative).  The  costs  associated  with  the 
implementation  of  classification  at  the  NBAFS  site  are  far  higher  than  the  costs  reported  for  other 
demonstration  projects.  This  is  a  direct  result  of  the  high  density  of  metallic  items  in  the  subsurface 
in  conjunction  with  attempting  to  resolve  challenging  TOI  (20-mm  projectiles).  Additional  factors 
impacting  the  cost  were  the  requirement  to  have  on-site  archaeological  oversight  and  the  high  rate 
of  QC  involvement  in  the  project.  The  costs  associated  with  the  NBAFS  demonstration  are  not 
typical  of  other  geophysical  classification  demonstrations  and  could  likely  be  considered  an 
outlier.  However,  the  results  of  the  NBAFS  demonstration  can  be  used  as  a  point  of  reference  for 
sites  with  similar  challenging  constraints. 
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Based  on  the  results  of  the  TOAR  demonstration,  the  reduction  in  the  number  of  clutter  items  that 
were  treated  as  potential  UXO  was  71  percent.  It  is  known  from  other  demonstrations  that  the  data 
collection  and  analysis  required  to  implement  classification  requires  more  resources  than  a 
conventional  detection  survey,  however,  these  costs  were  enhanced  at  TOAR.  At  TOAR,  data 
collection  production  rates  were  hindered  because  of  a  variety  of  challenging  conditions  including 
(but  not  limited  to)  dense  woods,  long  travel  times  to  and  from  the  site,  and  TEMTADS  software 
and  hardware  issues1.  These  factors  served  to  increase  the  costs  associated  with  data  collection 
and  data  processing.  However,  even  taking  into  consideration  these  added  up-front  costs  the 
savings  in  the  intrusive  phase  are  more  than  repaid.  Based  on  the  results  at  the  TOAR 
demonstration,  a  similar  reduction  (7 1  percent)  in  the  number  of  clutter  items  treated  as  potential 
UXO  should  be  possible  at  most  sites  with  similar  or  less  challenging  conditions. 


1  These  issues  are  discussed  in  more  detail  in  Section  9,  Implementation  Issues. 
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9.0  IMPLEMENTATION  ISSUES 


As  discussed  in  Section  8,  one  of  the  most  important  implementation  issues  related  to  the 
TEMTADS  technology  is  the  need  to  demonstrate  that  the  technology  can  significantly  reduce  the 
cost  of  remediation.  Even  with  the  reduction  in  the  costs  associated  with  intrusive  investigation  of 
non-hazardous  scrap,  the  cost  for  deploying  the  TEMTADS  will,  at  least  in  the  short  term,  remain 
significantly  higher  than  the  cost  for  commercially  available  conventional  technology  such  as  the 
Geonics  EM61-MK2. 

Additionally,  because  of  the  experimental  nature  of  the  TEMTADS  at  the  time  of  the 
demonstration,  the  sensor  is  not  considered  standard  commercial  off-the-shelf.  Because  of  the 
specialized  nature  and  lack  of  availability  of  the  technology,  time  was  lost  because  of  software  and 
hardware  issues  that  required  NRL  support  for  solutions  during  both  demonstrations.  The 
TEMTADS  can  be  deployed  in  the  field  using  personnel  with  the  same  technical  skills  as  those 
who  routinely  conduct  UXO-related  geophysical  mapping  surveys.  However,  data  processing 
tends  to  be  more  demanding  than  for  conventional  surveys. 

9.1  IMPLEMENTATION  ISSUES  SPECIFIC  TO  NBAFS 

The  underlying  premise  of  geophysical  classification  is  that  remediation  costs  can  be  reduced  by 
classifying  an  anomaly  source  as  TOI  or  non-TOI,  thus  limiting  intrusive  investigation  to  those 
anomalies  that  are  potentially  TOI.  This  premise  assumes  that  the  percentage  of  targets  that  are 
potentially  TOI  is  sufficiently  small  and  the  cost  savings  associated  with  reducing  the  number  of 
required  excavations  is  greater  than  the  cost  of  implementation  of  classification.  Using  generic 
assumptions  of  $50  per  anomaly  for  classification  (more  typical  than  the  costs  experienced  on  the 
NBAFS  project)  and  $200  per  excavation  (also  more  typical)  as  a  starting  point,  a  minimum  dig 
reduction  of  25  percent  is  required  to  offset  the  cost  of  classification  (obviously  any  benefit 
realized  would  require  more  than  a  25  percent  reduction  and  would  be  directly  proportional  to 
the  percent  reduction).  The  intrusive  results  at  NBAFS  indicate  that,  even  with  perfect 
classification,  the  maximum  dig  reduction  at  the  site  could  only  be  30  percent.  Clearly 
classification  does  not  offer  the  potential  for  cost  savings  at  this  site,  especially  if  20-mm 
projectiles  are  included  in  the  TOI  list.  A  20-mm  projectile  is  one  of  the  smallest  items  that  can  be 
reliably  detected  using  digital  geophysical  mapping  methods  and  therefore  has  a  low  SNR.  At 
NBAFS  there  were  many  20-mm  targets  throughout  the  site.  This  made  the  classification  problem 
especially  challenging  for  two  reasons 

•  The  high  percentage  of  multi-target  scenarios  where  signatures  overlapped 

•  The  size  of  scrap  fragments  at  NBAFS  were  often  similar  in  size  to  20-mm  projectiles 

In  addition  to  the  percent  of  TOI,  the  number  of  sources  encountered  for  each  anomaly 
significantly  impacted  the  usability  of  the  results.  Approximately  33  percent  of  the  anomalies 
investigated  resulted  in  more  than  three  sources  -  under  these  circumstances  the  inversion  results 
do  not  appear  to  accurately  represent  the  ground  truth.  Additionally,  there  currently  is  no  way  to 
use  the  inversion  results  to  reliably  determine  the  number  of  sources,  so  there  is  no  way  to  prove 
that  the  inversion  results  are  valid  when  the  anomaly  density  (and,  by  implication,  source  density) 
is  as  high  as  demonstrated  at  NBAFS. 
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9.2  IMPLEMENTATION  ISSUES  SPECIFIC  TO  TOAR 


While  the  RTS  can  be  used  for  positioning  in  open  areas  or  narrow  corridors,  the  TEMTADS  unit 
cannot  be  carried  in  such  a  way  that  it  is  always  moving  perpendicular  to  the  RTS  base  station, 
and  thus  avoid  being  shielded  from  the  base  by  the  operators.  In  order  to  facilitate  positioning  in  a 
heavily  wooded  area  using  RTS,  the  RTS  prism  located  above  the  TEMTADS  unit  had  to  be  raised 
above  the  heads  of  the  operators.  This  allowed  free  movement  by  the  operators  and  increased  the 
data  production  rate;  however,  the  added  height  of  the  prism  more  than  likely  caused  dynamic  IVS 
measurement  MQO  failures  because  of  the  oscillation  of  the  prism  while  walking.  Additionally, 
dynamic  data  with  a  lower  prism  height  were  not  collected,  so  it  is  not  possible  to  rule  out  that  just 
walking  in  litter  mode  with  the  RTS  was  not  the  issue.  Prior  experience  using  the  RTS  with  the 
TEMTADS  in  wheeled  mode  had  no  positioning  MQO  failures. 

Data  collection  production  rates  were  hindered  for  the  following  reasons: 

•  Approximately  1.5  hours  of  travel  time  was  needed  after  reaching  the  site  each  day  to 
access  the  collection  grids. 

•  Daily  reassembly  and  breakdown  were  required,  as  there  was  no  secure  storage  facility.  A 
temporary  carport  was  installed  for  a  partial  breakdown  area;  however,  cables  needed  to 
be  secured  nightly  because  of  animal  activity. 

•  Setting  up  the  RTS  base  stations  in  multiple  locations  took  more  time  than  a  standard  RTK 
GPS  set  up.  There  was  also  a  steep  learning  curve  for  speedy  assembly  of  the  RTS;  this 
often  delayed  the  start  of  production. 

Because  of  the  experimental  nature  of  the  TEMTADS,  several  days  of  production  were  lost 
because  of  software  issues  and  a  hard  drive  failure,  which  resulted  in  multiple  trips  to  NRL  for 
solutions. 
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10.0  CONCLUSIONS 


CH2M  performed  ESTCP  MR  Live  Site  Demonstrations  at  NBAFS,  NH,  in  the  summer  of  2013, 
and  at  MRS  R-04A  West  site  at  TOAR  FUDS,  Pennsylvania,  in  the  summer  of  2015.  The 
demonstrations  involved  the  use  of  the  TEMTADS  EMI  sensor  for  classification  at  sites  designed 
to  challenge  the  technology  by: 

•  Investigating  the  classification  methodology  at  a  site  that  was  suspected  to  contain  both  a 
high  density  of  subsurface  metal  as  well  as  a  large  variety  of  munitions  down  to  20-mm 
projectiles  (NBAFS) 

•  Investigating  the  efficacy  of  the  sensor  at  a  densely  wooded  site  with  challenging 
micro-terrain  features  (e.g.,  impact  craters,  rocks,  boulders,  gullies)  for  detection  of 
munitions  down  to  the  size  of  37-mm  projectiles  (TOAR) 

The  NRL  TEMTADS  was  demonstrated  in  both  dynamic  and  cued  modes.  It  was  deployed  on  its 
standard  wheels  at  NBAFS  and  in  a  two-person  litter  configuration  at  TOAR.  At  NBAFS 
positioning  was  achieved  with  a  RTK  GPS  and  at  TOAR  with  a  Trimble  RTS. 

At  NBAFS,  approximately  6. 1  acres  were  mapped  with  the  TEMTADS.  Evaluation  of  the  dynamic 
data  resulted  in  the  selection  of  18,373  targets.  As  the  density  of  geophysical  anomalies  identified 
was  an  order  of  magnitude  greater  than  anticipated,  a  subset  area  of  the  site  was  chosen  for  cued 
investigation  and  1,500  anomalies  were  identified  for  cued  interrogation.  The  demonstration  was 
halted  several  weeks  into  the  dig  program  as  it  was  determined  that  use  of  AGC  technology  at  this 
site  would  not  result  in  cost  savings  and  would  not  be  effective  with  respect  to  classification. 
Intrusive  investigations  yielded  TOI  at  70  percent  of  each  dig  location.  Additionally, 
approximately  33  percent  of  the  anomalies  investigated  resulted  in  the  recovery  of  three  or  more 
sources  and  under  these  circumstances  the  inversion  results  do  not  appear  to  accurately  represent 
the  ground  truth.  These  results  indicated  that  submission  of  a  final  prioritized  list  would  add  no 
value  to  the  demonstration.  Additionally,  given  the  failure  of  the  derived  signatures  to  accurately 
predict  the  anomaly  sources  when  more  than  3  sources  were  recovered  and  the  high  percentage  of 
multi-target  scenarios,  an  analysis  of  the  system’s  performance  against  the  few  seeds  on  the  cued 
interrogation  list  was  not  undertaken  as  added  no  value  to  the  demonstration. 

At  TOAR,  approximately  0.71  acre  was  dynamically  surveyed  with  the  TEMTADS  system. 
Production  was  significantly  hindered  by  the  remote  location  and  site  conditions,  and  only  one  of 
the  four  grids  initially  selected  for  investigation  achieved  100  percent  coverage  (not  including  gaps 
because  of  physical  obstructions).  Based  upon  this  effort,  production  rates  under  less  challenging 
conditions  are  estimated  to  be  much  higher.  A  total  of  429  anomalies  were  identified  by  the 
TEMTADS  dynamic  data  analysis.  These  anomaly  locations  and  an  additional  68  targets  selected 
by  the  MPV  survey  were  interrogated  (cued  data  collection)  and  classified  with  the  TEMTADS 
system.  The  targets  were  subsequently  classified  as  being  a  potential  TOI  (dig)  or  high-likelihood 
non-TOI  (do  not  dig).  All  known  TOI  (seeds  and  native)  were  successfully  detected  and  classified. 
From  the  perspective  of  demonstrating  the  potential  for  cost  savings  and  risk  reduction,  the  TOAR 
study  was  successful.  The  data  analyst  was  able  to  correctly  identify  all  the  munitions  on  the 
anomaly  list  with  an  approximately  22  percent  rate  of  false  positives.  This  resulted  in  a  71  percent 
reduction  in  the  number  of  intrusive  investigations  necessary. 
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